May 28 17:07:01.003741 ip-10-0-130-46 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 28 17:07:01.003754 ip-10-0-130-46 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 28 17:07:01.003762 ip-10-0-130-46 systemd[1]: kubelet.service: Failed with result 'resources'. May 28 17:07:01.004070 ip-10-0-130-46 systemd[1]: Failed to start Kubernetes Kubelet. May 28 17:07:11.180118 ip-10-0-130-46 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 28 17:07:11.180137 ip-10-0-130-46 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b39d8ca8005f4008a549c1d19f52e7fe -- May 28 17:09:49.975250 ip-10-0-130-46 systemd[1]: Starting Kubernetes Kubelet... May 28 17:09:50.391073 ip-10-0-130-46 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 17:09:50.391073 ip-10-0-130-46 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 28 17:09:50.391073 ip-10-0-130-46 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 17:09:50.391073 ip-10-0-130-46 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 28 17:09:50.391073 ip-10-0-130-46 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 17:09:50.401249 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.401151 2566 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 28 17:09:50.407671 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407645 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 17:09:50.407671 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407664 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 17:09:50.407671 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407667 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 17:09:50.407671 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407671 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 17:09:50.407671 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407674 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 17:09:50.407671 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407676 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 17:09:50.407671 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407679 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 17:09:50.407671 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407682 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407684 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407687 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407690 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407692 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407696 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407699 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407701 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407704 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407706 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407709 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407711 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407714 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407716 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407719 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407722 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407724 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407727 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407730 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407733 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 17:09:50.407956 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407737 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407739 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407742 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407744 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407747 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407749 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407752 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407754 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407757 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407759 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407762 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407765 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407770 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407773 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407776 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407779 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407782 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407784 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407786 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407789 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 17:09:50.408440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407792 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407794 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407797 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407800 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407802 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407805 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407808 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407811 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407813 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407816 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407818 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407821 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407824 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407826 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407829 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407832 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407834 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407837 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407839 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407842 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 17:09:50.408994 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407844 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407846 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407849 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407854 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407858 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407861 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407865 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407868 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407871 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407874 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407877 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407880 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407883 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407885 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407888 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407891 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407894 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407897 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.407899 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 17:09:50.409477 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408298 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408304 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408306 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408309 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408312 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408315 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408317 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408320 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408322 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408325 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408327 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408330 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408333 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408336 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408339 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408341 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408344 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408347 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408350 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408353 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 17:09:50.409954 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408356 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408359 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408361 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408364 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408368 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408370 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408373 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408375 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408378 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408380 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408383 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408386 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408388 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408391 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408393 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408396 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408398 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408402 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408405 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408408 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 17:09:50.410442 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408410 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408413 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408415 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408418 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408420 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408423 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408425 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408428 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408431 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408434 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408436 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408440 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408443 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408446 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408450 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408454 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408457 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408461 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408465 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408468 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 17:09:50.410938 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408471 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408474 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408477 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408480 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408482 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408485 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408487 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408490 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408492 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408495 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408497 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408500 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408503 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408505 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408508 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408511 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408513 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408516 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408518 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 17:09:50.411451 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408521 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408524 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408527 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408530 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408533 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408535 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.408538 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409096 2566 flags.go:64] FLAG: --address="0.0.0.0" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409105 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409111 2566 flags.go:64] FLAG: --anonymous-auth="true" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409116 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409122 2566 flags.go:64] FLAG: --authentication-token-webhook="false" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409126 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409130 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409134 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409138 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409141 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409144 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409148 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409151 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409154 2566 flags.go:64] FLAG: --cgroup-root="" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409157 2566 flags.go:64] FLAG: --cgroups-per-qos="true" May 28 17:09:50.411913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409160 2566 flags.go:64] FLAG: --client-ca-file="" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409163 2566 flags.go:64] FLAG: --cloud-config="" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409166 2566 flags.go:64] FLAG: --cloud-provider="external" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409169 2566 flags.go:64] FLAG: --cluster-dns="[]" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409175 2566 flags.go:64] FLAG: --cluster-domain="" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409178 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409181 2566 flags.go:64] FLAG: --config-dir="" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409184 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409187 2566 flags.go:64] FLAG: --container-log-max-files="5" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409192 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409195 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409198 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409202 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409205 2566 flags.go:64] FLAG: --contention-profiling="false" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409208 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409211 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409214 2566 flags.go:64] FLAG: --cpu-manager-policy="none" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409217 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409221 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409224 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409227 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409230 2566 flags.go:64] FLAG: --enable-load-reader="false" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409234 2566 flags.go:64] FLAG: --enable-server="true" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409237 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409241 2566 flags.go:64] FLAG: --event-burst="100" May 28 17:09:50.412475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409244 2566 flags.go:64] FLAG: --event-qps="50" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409247 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409250 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409253 2566 flags.go:64] FLAG: --eviction-hard="" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409257 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409260 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409263 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409266 2566 flags.go:64] FLAG: --eviction-soft="" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409269 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409272 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409275 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409278 2566 flags.go:64] FLAG: --experimental-mounter-path="" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409281 2566 flags.go:64] FLAG: --fail-cgroupv1="false" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409284 2566 flags.go:64] FLAG: --fail-swap-on="true" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409287 2566 flags.go:64] FLAG: --feature-gates="" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409291 2566 flags.go:64] FLAG: --file-check-frequency="20s" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409294 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409298 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409301 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409304 2566 flags.go:64] FLAG: --healthz-port="10248" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409307 2566 flags.go:64] FLAG: --help="false" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409310 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-130-46.ec2.internal" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409313 2566 flags.go:64] FLAG: --housekeeping-interval="10s" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409316 2566 flags.go:64] FLAG: --http-check-frequency="20s" May 28 17:09:50.413109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409319 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409323 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409326 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409329 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409332 2566 flags.go:64] FLAG: --image-service-endpoint="" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409335 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409338 2566 flags.go:64] FLAG: --kube-api-burst="100" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409342 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409345 2566 flags.go:64] FLAG: --kube-api-qps="50" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409348 2566 flags.go:64] FLAG: --kube-reserved="" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409350 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409353 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409356 2566 flags.go:64] FLAG: --kubelet-cgroups="" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409359 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409362 2566 flags.go:64] FLAG: --lock-file="" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409364 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409367 2566 flags.go:64] FLAG: --log-flush-frequency="5s" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409371 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409376 2566 flags.go:64] FLAG: --log-json-split-stream="false" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409379 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409382 2566 flags.go:64] FLAG: --log-text-split-stream="false" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409384 2566 flags.go:64] FLAG: --logging-format="text" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409387 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409391 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" May 28 17:09:50.413692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409394 2566 flags.go:64] FLAG: --manifest-url="" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409413 2566 flags.go:64] FLAG: --manifest-url-header="" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409419 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409422 2566 flags.go:64] FLAG: --max-open-files="1000000" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409429 2566 flags.go:64] FLAG: --max-pods="110" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409432 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409435 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409438 2566 flags.go:64] FLAG: --memory-manager-policy="None" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409441 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409444 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409447 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409450 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409457 2566 flags.go:64] FLAG: --node-status-max-images="50" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409460 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409463 2566 flags.go:64] FLAG: --oom-score-adj="-999" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409467 2566 flags.go:64] FLAG: --pod-cidr="" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409470 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409476 2566 flags.go:64] FLAG: --pod-manifest-path="" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409479 2566 flags.go:64] FLAG: --pod-max-pids="-1" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409482 2566 flags.go:64] FLAG: --pods-per-core="0" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409486 2566 flags.go:64] FLAG: --port="10250" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409489 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409492 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-051c8e3115f9dcfca" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409495 2566 flags.go:64] FLAG: --qos-reserved="" May 28 17:09:50.414332 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409498 2566 flags.go:64] FLAG: --read-only-port="10255" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409501 2566 flags.go:64] FLAG: --register-node="true" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409504 2566 flags.go:64] FLAG: --register-schedulable="true" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409506 2566 flags.go:64] FLAG: --register-with-taints="" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409510 2566 flags.go:64] FLAG: --registry-burst="10" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409513 2566 flags.go:64] FLAG: --registry-qps="5" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409516 2566 flags.go:64] FLAG: --reserved-cpus="" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409519 2566 flags.go:64] FLAG: --reserved-memory="" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409523 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409526 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409529 2566 flags.go:64] FLAG: --rotate-certificates="false" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409532 2566 flags.go:64] FLAG: --rotate-server-certificates="false" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409535 2566 flags.go:64] FLAG: --runonce="false" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409538 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409541 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409544 2566 flags.go:64] FLAG: --seccomp-default="false" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409547 2566 flags.go:64] FLAG: --serialize-image-pulls="true" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409549 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409552 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409555 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409558 2566 flags.go:64] FLAG: --storage-driver-password="root" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409561 2566 flags.go:64] FLAG: --storage-driver-secure="false" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409564 2566 flags.go:64] FLAG: --storage-driver-table="stats" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409567 2566 flags.go:64] FLAG: --storage-driver-user="root" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409571 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409574 2566 flags.go:64] FLAG: --sync-frequency="1m0s" May 28 17:09:50.414906 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409582 2566 flags.go:64] FLAG: --system-cgroups="" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409585 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409590 2566 flags.go:64] FLAG: --system-reserved-cgroup="" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409593 2566 flags.go:64] FLAG: --tls-cert-file="" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409596 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409600 2566 flags.go:64] FLAG: --tls-min-version="" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409603 2566 flags.go:64] FLAG: --tls-private-key-file="" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409606 2566 flags.go:64] FLAG: --topology-manager-policy="none" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409609 2566 flags.go:64] FLAG: --topology-manager-policy-options="" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409612 2566 flags.go:64] FLAG: --topology-manager-scope="container" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409615 2566 flags.go:64] FLAG: --v="2" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409620 2566 flags.go:64] FLAG: --version="false" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409624 2566 flags.go:64] FLAG: --vmodule="" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409628 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.409631 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409723 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409727 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409730 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409733 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409736 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409738 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409741 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409744 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 17:09:50.415574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409747 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409750 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409752 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409755 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409757 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409760 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409762 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409765 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409769 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409774 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409777 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409780 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409782 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409785 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409787 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409790 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409793 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409795 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409798 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 17:09:50.416174 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409800 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409803 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409806 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409808 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409811 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409813 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409816 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409818 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409821 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409823 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409826 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409828 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409830 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409833 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409837 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409840 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409843 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409846 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409849 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 17:09:50.416662 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409851 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409854 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409856 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409861 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409864 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409866 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409869 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409871 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409874 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409876 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409880 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409883 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409885 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409888 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409890 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409893 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409896 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409898 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409901 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409904 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 17:09:50.417145 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409906 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409909 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409912 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409915 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409917 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409920 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409923 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409925 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409928 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409931 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409933 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409936 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409939 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409957 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409960 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409963 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409967 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409970 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409973 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 17:09:50.417638 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.409976 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.411194 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.417243 2566 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.417259 2566 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417306 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417311 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417314 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417318 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417321 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417324 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417327 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417329 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417332 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417334 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417337 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417339 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 17:09:50.418099 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417342 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417345 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417348 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417351 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417353 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417356 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417358 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417361 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417364 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417366 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417369 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417372 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417374 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417377 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417379 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417382 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417384 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417388 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417390 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417394 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 17:09:50.418495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417397 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417400 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417403 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417405 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417408 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417411 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417413 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417416 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417419 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417421 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417423 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417426 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417429 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417432 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417434 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417437 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417439 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417442 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417445 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417447 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 17:09:50.419019 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417450 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417452 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417455 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417457 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417460 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417464 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417467 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417470 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417472 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417475 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417478 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417481 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417485 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417488 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417490 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417493 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417497 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417501 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417504 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 17:09:50.419495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417507 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417510 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417512 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417515 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417518 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417520 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417523 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417525 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417528 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417530 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417533 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417536 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417538 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417541 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417543 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.417549 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 17:09:50.419975 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417644 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417649 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417652 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417656 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417660 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417663 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417666 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417669 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417672 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417675 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417678 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417681 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417684 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417687 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417689 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417692 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417695 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417697 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417700 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 17:09:50.420373 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417703 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417706 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417709 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417711 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417714 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417716 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417719 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417721 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417724 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417727 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417729 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417732 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417734 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417737 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417739 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417742 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417745 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417747 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417749 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417752 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 17:09:50.420933 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417754 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417757 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417760 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417762 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417765 2566 feature_gate.go:328] unrecognized feature gate: Example2 May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417768 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417770 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417773 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417776 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417778 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417781 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417783 2566 feature_gate.go:328] unrecognized feature gate: Example May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417786 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417788 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417791 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417793 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417796 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417798 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417802 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417806 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 17:09:50.421440 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417808 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417811 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417813 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417816 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417818 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417821 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417823 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417825 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417828 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417830 2566 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417833 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417835 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417838 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417841 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417843 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417846 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417848 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417851 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417854 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 17:09:50.421935 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417856 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417859 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417861 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417863 2566 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417866 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417868 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417871 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:50.417873 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.417878 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.418650 2566 server.go:962] "Client rotation is on, will bootstrap in background" May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.420468 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.421363 2566 server.go:1019] "Starting client certificate rotation" May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.421457 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 17:09:50.422450 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.421493 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 17:09:50.446128 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.446107 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 17:09:50.450197 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.450182 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 17:09:50.470215 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.470196 2566 log.go:25] "Validated CRI v1 runtime API" May 28 17:09:50.474634 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.474614 2566 log.go:25] "Validated CRI v1 image API" May 28 17:09:50.475798 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.475783 2566 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 28 17:09:50.478128 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.478112 2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 17:09:50.478228 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.478210 2566 fs.go:135] Filesystem UUIDs: map[3183261b-02dc-41d6-918d-4df2e7931e09:/dev/nvme0n1p3 47719a3d-a482-4400-8fd7-1f63045d5fb9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] May 28 17:09:50.478269 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.478230 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 28 17:09:50.482434 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.482330 2566 manager.go:217] Machine: {Timestamp:2026-05-28 17:09:50.481811384 +0000 UTC m=+0.394204030 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100183 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27948f371c2c74788e9e08066d158b SystemUUID:ec27948f-371c-2c74-788e-9e08066d158b BootID:b39d8ca8-005f-4008-a549-c1d19f52e7fe Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:94:b6:0b:0d:5b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:94:b6:0b:0d:5b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:17:9f:21:4e:66 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 28 17:09:50.483192 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.483181 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 28 17:09:50.483304 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.483292 2566 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 28 17:09:50.486256 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.486229 2566 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 28 17:09:50.486401 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.486259 2566 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-46.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 28 17:09:50.486445 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.486412 2566 topology_manager.go:138] "Creating topology manager with none policy" May 28 17:09:50.486445 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.486421 2566 container_manager_linux.go:306] "Creating device plugin manager" May 28 17:09:50.486445 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.486428 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 17:09:50.487230 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.487219 2566 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 17:09:50.488979 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.488969 2566 state_mem.go:36] "Initialized new in-memory state store" May 28 17:09:50.489093 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.489084 2566 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 28 17:09:50.491354 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.491345 2566 kubelet.go:491] "Attempting to sync node with API server" May 28 17:09:50.491389 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.491359 2566 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 28 17:09:50.491389 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.491373 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 28 17:09:50.491389 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.491382 2566 kubelet.go:397] "Adding apiserver pod source" May 28 17:09:50.491389 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.491390 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 28 17:09:50.492501 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.492489 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 17:09:50.492549 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.492508 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 17:09:50.495574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.495549 2566 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 28 17:09:50.497285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.497271 2566 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 28 17:09:50.498202 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498189 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 28 17:09:50.498242 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498211 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 28 17:09:50.498242 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498223 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 28 17:09:50.498242 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498233 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 28 17:09:50.498328 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498243 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 28 17:09:50.498328 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498254 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 28 17:09:50.498328 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498263 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 28 17:09:50.498328 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498271 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 28 17:09:50.498328 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498280 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 28 17:09:50.498328 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498288 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 28 17:09:50.498328 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498309 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 28 17:09:50.498516 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.498501 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 28 17:09:50.499471 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.499459 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 28 17:09:50.499506 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.499473 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 28 17:09:50.499907 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.499890 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tc25g" May 28 17:09:50.501459 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.501431 2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-46.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 28 17:09:50.501459 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.501444 2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 28 17:09:50.502757 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.502744 2566 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 28 17:09:50.502808 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.502784 2566 server.go:1295] "Started kubelet" May 28 17:09:50.503457 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.503399 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 28 17:09:50.504356 ip-10-0-130-46 systemd[1]: Started Kubernetes Kubelet. May 28 17:09:50.504937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.504347 2566 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 28 17:09:50.504937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.504413 2566 server_v1.go:47] "podresources" method="list" useActivePods=true May 28 17:09:50.506530 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.505829 2566 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 28 17:09:50.507171 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.507152 2566 server.go:317] "Adding debug handlers to kubelet server" May 28 17:09:50.507659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.507622 2566 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-46.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 28 17:09:50.510446 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.510425 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tc25g" May 28 17:09:50.513181 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.513164 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 28 17:09:50.513181 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.513171 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 28 17:09:50.513352 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.513237 2566 volume_manager.go:295] "The desired_state_of_world populator starts" May 28 17:09:50.513352 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.513248 2566 volume_manager.go:297] "Starting Kubelet Volume Manager" May 28 17:09:50.513352 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.513288 2566 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 28 17:09:50.513352 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.513339 2566 reconstruct.go:97] "Volume reconstruction finished" May 28 17:09:50.513352 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.513348 2566 reconciler.go:26] "Reconciler: start to sync state" May 28 17:09:50.513758 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.513734 2566 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 28 17:09:50.514129 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.514095 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:50.514616 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.514599 2566 factory.go:55] Registering systemd factory May 28 17:09:50.514686 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.514628 2566 factory.go:223] Registration of the systemd container factory successfully May 28 17:09:50.514844 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.514830 2566 factory.go:153] Registering CRI-O factory May 28 17:09:50.514883 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.514848 2566 factory.go:223] Registration of the crio container factory successfully May 28 17:09:50.514913 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.514901 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 28 17:09:50.514964 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.514921 2566 factory.go:103] Registering Raw factory May 28 17:09:50.514964 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.514931 2566 manager.go:1196] Started watching for new ooms in manager May 28 17:09:50.515333 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.515319 2566 manager.go:319] Starting recovery of all containers May 28 17:09:50.517562 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.517543 2566 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 28 17:09:50.521618 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.521507 2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-46.ec2.internal\" not found" node="ip-10-0-130-46.ec2.internal" May 28 17:09:50.524705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.524690 2566 manager.go:324] Recovery completed May 28 17:09:50.528626 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.528614 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 17:09:50.530561 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.530548 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientMemory" May 28 17:09:50.530627 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.530573 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasNoDiskPressure" May 28 17:09:50.530627 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.530583 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientPID" May 28 17:09:50.531519 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.531506 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" May 28 17:09:50.531583 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.531518 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 28 17:09:50.531583 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.531541 2566 state_mem.go:36] "Initialized new in-memory state store" May 28 17:09:50.533735 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.533722 2566 policy_none.go:49] "None policy: Start" May 28 17:09:50.533790 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.533739 2566 memory_manager.go:186] "Starting memorymanager" policy="None" May 28 17:09:50.533790 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.533750 2566 state_mem.go:35] "Initializing new in-memory state store" May 28 17:09:50.573519 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.573507 2566 manager.go:341] "Starting Device Plugin manager" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.573533 2566 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.573542 2566 server.go:85] "Starting device plugin registration server" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.573728 2566 eviction_manager.go:189] "Eviction manager: starting control loop" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.573739 2566 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.573899 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.573988 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.573999 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.574295 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 28 17:09:50.576652 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.574326 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:50.635602 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.635581 2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 28 17:09:50.636669 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.636653 2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 28 17:09:50.636746 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.636680 2566 status_manager.go:230] "Starting to sync pod status with apiserver" May 28 17:09:50.636746 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.636697 2566 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 28 17:09:50.636746 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.636704 2566 kubelet.go:2451] "Starting kubelet main sync loop" May 28 17:09:50.636746 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.636736 2566 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 28 17:09:50.640109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.640087 2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 17:09:50.674497 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.674451 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 17:09:50.675345 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.675329 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientMemory" May 28 17:09:50.675411 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.675359 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasNoDiskPressure" May 28 17:09:50.675411 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.675369 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientPID" May 28 17:09:50.675411 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.675392 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-46.ec2.internal" May 28 17:09:50.684023 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.684006 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-46.ec2.internal" May 28 17:09:50.684078 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.684027 2566 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-46.ec2.internal\": node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:50.698995 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.698975 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:50.737191 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.737165 2566 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal"] May 28 17:09:50.737264 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.737241 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 17:09:50.738193 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.738179 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientMemory" May 28 17:09:50.738267 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.738210 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasNoDiskPressure" May 28 17:09:50.738267 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.738225 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientPID" May 28 17:09:50.739613 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.739600 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 17:09:50.739790 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.739774 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:50.739831 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.739806 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 17:09:50.740373 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.740357 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientMemory" May 28 17:09:50.740428 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.740386 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasNoDiskPressure" May 28 17:09:50.740428 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.740358 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientMemory" May 28 17:09:50.740428 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.740422 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasNoDiskPressure" May 28 17:09:50.740525 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.740433 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientPID" May 28 17:09:50.740525 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.740398 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientPID" May 28 17:09:50.741465 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.741448 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" May 28 17:09:50.741510 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.741482 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 17:09:50.742141 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.742128 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientMemory" May 28 17:09:50.742203 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.742153 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasNoDiskPressure" May 28 17:09:50.742203 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.742163 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeHasSufficientPID" May 28 17:09:50.765470 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.765452 2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-46.ec2.internal\" not found" node="ip-10-0-130-46.ec2.internal" May 28 17:09:50.769640 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.769624 2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-46.ec2.internal\" not found" node="ip-10-0-130-46.ec2.internal" May 28 17:09:50.799735 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.799717 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:50.899849 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:50.899814 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:50.915153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.915133 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14867644809a4eb2323ebb06f498f120-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal\" (UID: \"14867644809a4eb2323ebb06f498f120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:50.915207 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.915160 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14867644809a4eb2323ebb06f498f120-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal\" (UID: \"14867644809a4eb2323ebb06f498f120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:50.915207 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:50.915180 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5866c7b02ae35c0d688802a9a7d31ed6-config\") pod \"kube-apiserver-proxy-ip-10-0-130-46.ec2.internal\" (UID: \"5866c7b02ae35c0d688802a9a7d31ed6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" May 28 17:09:51.000611 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.000551 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:51.015866 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.015842 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14867644809a4eb2323ebb06f498f120-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal\" (UID: \"14867644809a4eb2323ebb06f498f120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:51.015910 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.015875 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14867644809a4eb2323ebb06f498f120-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal\" (UID: \"14867644809a4eb2323ebb06f498f120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:51.015910 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.015890 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5866c7b02ae35c0d688802a9a7d31ed6-config\") pod \"kube-apiserver-proxy-ip-10-0-130-46.ec2.internal\" (UID: \"5866c7b02ae35c0d688802a9a7d31ed6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" May 28 17:09:51.015989 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.015955 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/14867644809a4eb2323ebb06f498f120-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal\" (UID: \"14867644809a4eb2323ebb06f498f120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:51.015989 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.015968 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14867644809a4eb2323ebb06f498f120-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal\" (UID: \"14867644809a4eb2323ebb06f498f120\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:51.015989 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.015930 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5866c7b02ae35c0d688802a9a7d31ed6-config\") pod \"kube-apiserver-proxy-ip-10-0-130-46.ec2.internal\" (UID: \"5866c7b02ae35c0d688802a9a7d31ed6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" May 28 17:09:51.068028 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.067989 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:51.071728 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.071712 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" May 28 17:09:51.101264 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.101234 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:51.201711 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.201679 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:51.302207 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.302124 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:51.402623 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.402588 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-46.ec2.internal\" not found" May 28 17:09:51.416606 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.416588 2566 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 28 17:09:51.420904 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.420892 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 28 17:09:51.421028 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.421014 2566 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 28 17:09:51.421067 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.421051 2566 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 28 17:09:51.421098 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.421066 2566 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 28 17:09:51.492264 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.492222 2566 apiserver.go:52] "Watching apiserver" May 28 17:09:51.509716 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.509691 2566 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 28 17:09:51.510036 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.510016 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj","openshift-cluster-node-tuning-operator/tuned-z4h74","openshift-dns/node-resolver-4frtv","openshift-image-registry/node-ca-szzhz","openshift-multus/multus-additional-cni-plugins-vlvsd","openshift-multus/multus-dgbcd","openshift-multus/network-metrics-daemon-lsl89","openshift-network-diagnostics/network-check-target-kxlwn","kube-system/konnectivity-agent-g99r4","openshift-network-operator/iptables-alerter-xnpdv","openshift-ovn-kubernetes/ovnkube-node-dnjtt"] May 28 17:09:51.512314 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.512288 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 17:04:50 +0000 UTC" deadline="2027-11-01 09:58:36.253566163 +0000 UTC" May 28 17:09:51.512314 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.512310 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12520h48m44.741258549s" May 28 17:09:51.512453 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.512359 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.513256 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.513242 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 28 17:09:51.513330 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.513280 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.513547 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.513508 2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" May 28 17:09:51.514298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.514268 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.514428 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.514363 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.515462 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.515434 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 28 17:09:51.515614 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.515594 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 28 17:09:51.515681 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.515636 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 28 17:09:51.515858 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.515835 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cnt49\"" May 28 17:09:51.515977 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.515939 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-brvct\"" May 28 17:09:51.516120 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.516104 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 28 17:09:51.516214 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.516201 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 28 17:09:51.517147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.517126 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.517816 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.517795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a73de68c-3175-4736-bf34-52f955c17308-serviceca\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.517886 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.517833 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-sys-fs\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.517925 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.517885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-lib-modules\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.517925 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.517907 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-tuned\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518013 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.517927 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-hosts-file\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.518013 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.517958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-tmp-dir\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.518013 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.517981 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.518013 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-device-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518017 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysconfig\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-run\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518046 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-sys\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518060 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzw55\" (UniqueName: \"kubernetes.io/projected/a73de68c-3175-4736-bf34-52f955c17308-kube-api-access-jzw55\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518075 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-registration-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518088 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-host\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a73de68c-3175-4736-bf34-52f955c17308-host\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-etc-selinux\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.518147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l824h\" (UniqueName: \"kubernetes.io/projected/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-kube-api-access-l824h\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518171 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-systemd\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-var-lib-kubelet\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518227 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr6cv\" (UniqueName: \"kubernetes.io/projected/e644e105-0aef-40d5-ac36-9ce70a2284b4-kube-api-access-kr6cv\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518250 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-kubernetes\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518266 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysctl-d\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518272 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dgbcd" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518280 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysctl-conf\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xs96\" (UniqueName: \"kubernetes.io/projected/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-kube-api-access-4xs96\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518339 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-socket-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518362 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-modprobe-d\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.518474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.518383 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e644e105-0aef-40d5-ac36-9ce70a2284b4-tmp\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.519325 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.519308 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:51.519411 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.519390 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:09:51.520455 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.520431 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:51.520534 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.520489 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:09:51.520534 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.520501 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g99r4" May 28 17:09:51.520878 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.520862 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 28 17:09:51.521178 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521156 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 28 17:09:51.521439 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521426 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 28 17:09:51.521509 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521451 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 28 17:09:51.521559 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521526 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 28 17:09:51.521608 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521578 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5rrpq\"" May 28 17:09:51.521608 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521585 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 28 17:09:51.521866 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521739 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kbrwr\"" May 28 17:09:51.521866 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521767 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 28 17:09:51.521866 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521774 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 28 17:09:51.521866 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521793 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wv65s\"" May 28 17:09:51.521866 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521833 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 28 17:09:51.521866 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.521778 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 28 17:09:51.522150 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.522022 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.522150 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.522044 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7bs92\"" May 28 17:09:51.522150 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.522082 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 28 17:09:51.523077 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.523049 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 28 17:09:51.523187 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.523079 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 28 17:09:51.523601 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.523588 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.523894 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.523878 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-skhb2\"" May 28 17:09:51.526322 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526308 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 28 17:09:51.526393 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526322 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 28 17:09:51.526393 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526331 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 28 17:09:51.526674 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526657 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 28 17:09:51.526772 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526700 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 28 17:09:51.526772 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526700 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 28 17:09:51.526772 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526750 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l9mxl\"" May 28 17:09:51.526772 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526705 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 28 17:09:51.526960 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526788 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9vt8g\"" May 28 17:09:51.526960 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526866 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 28 17:09:51.527024 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.526939 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 28 17:09:51.529363 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.529348 2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 17:09:51.531033 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.531020 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 17:09:51.531091 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.531081 2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" May 28 17:09:51.531659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.531646 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal"] May 28 17:09:51.539965 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.539935 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 17:09:51.540153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.540137 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal"] May 28 17:09:51.550087 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.550066 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hgwxk" May 28 17:09:51.559289 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.559249 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hgwxk" May 28 17:09:51.614292 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.614272 2566 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 28 17:09:51.619224 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619200 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-run-netns\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.619341 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619231 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzw55\" (UniqueName: \"kubernetes.io/projected/a73de68c-3175-4736-bf34-52f955c17308-kube-api-access-jzw55\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.619341 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovn-node-metrics-cert\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.619341 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619273 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-cni-binary-copy\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.619341 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619289 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:51.619341 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619304 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-systemd\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.619341 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619327 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-var-lib-kubelet\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619346 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kr6cv\" (UniqueName: \"kubernetes.io/projected/e644e105-0aef-40d5-ac36-9ce70a2284b4-kube-api-access-kr6cv\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619370 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-etc-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-cni-bin\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619395 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-systemd\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619418 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abe63baf-7676-4084-9037-525c520b3707-host-slash\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619422 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-var-lib-kubelet\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619444 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-os-release\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619465 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619483 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj9mz\" (UniqueName: \"kubernetes.io/projected/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-kube-api-access-xj9mz\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619497 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-cnibin\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619529 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-kubernetes\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619584 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-kubernetes\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619592 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysctl-d\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.619619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619612 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4xs96\" (UniqueName: \"kubernetes.io/projected/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-kube-api-access-4xs96\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619629 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-ovn\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619644 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5sd8\" (UniqueName: \"kubernetes.io/projected/e7653292-2abd-4283-b387-3ad935b2be56-kube-api-access-q5sd8\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619668 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abe63baf-7676-4084-9037-525c520b3707-iptables-alerter-script\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619683 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdqvg\" (UniqueName: \"kubernetes.io/projected/abe63baf-7676-4084-9037-525c520b3707-kube-api-access-bdqvg\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619699 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-socket-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619718 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-modprobe-d\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619766 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e644e105-0aef-40d5-ac36-9ce70a2284b4-tmp\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619771 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysctl-d\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619796 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-cni-multus\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619824 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovnkube-config\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619830 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-socket-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619850 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619878 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-sys-fs\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619848 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-modprobe-d\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619902 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-tuned\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619940 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-hosts-file\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.620092 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619975 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-tmp-dir\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619977 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-sys-fs\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.619993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-kubelet\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620024 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620052 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-device-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620097 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-device-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-hosts-file\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620110 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysconfig\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620149 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-run\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620097 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620174 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-sys\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-multus-certs\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620216 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-run\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620208 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-sys\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620232 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-env-overrides\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620150 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysconfig\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620098 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620257 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-os-release\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.620817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620285 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-tmp-dir\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-systemd-units\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620354 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-registration-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620369 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-host\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620389 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-system-cni-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620422 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-host\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620426 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc76295c-2e54-4682-a401-5c84d779932a-cni-binary-copy\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620465 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-registration-dir\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620492 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-socket-dir-parent\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620513 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-k8s-cni-cncf-io\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620558 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a73de68c-3175-4736-bf34-52f955c17308-host\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620603 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a73de68c-3175-4736-bf34-52f955c17308-host\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620608 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-etc-selinux\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620643 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l824h\" (UniqueName: \"kubernetes.io/projected/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-kube-api-access-l824h\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-cni-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620684 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-etc-selinux\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.621472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-hostroot\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620725 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-conf-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620751 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-cni-netd\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620773 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-node-log\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d8b08027-e6a6-4103-85bd-8ac9b940b103-konnectivity-ca\") pod \"konnectivity-agent-g99r4\" (UID: \"d8b08027-e6a6-4103-85bd-8ac9b940b103\") " pod="kube-system/konnectivity-agent-g99r4" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620824 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysctl-conf\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620847 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-cnibin\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620869 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-var-lib-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620895 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-kubelet\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc76295c-2e54-4682-a401-5c84d779932a-multus-daemon-config\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-systemd\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620955 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-sysctl-conf\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.620991 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8gnl\" (UniqueName: \"kubernetes.io/projected/e04539f3-03d0-41ac-8c13-0f67edc8e25c-kube-api-access-p8gnl\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621017 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-etc-kubernetes\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621034 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-slash\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-log-socket\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621100 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovnkube-script-lib\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.622014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621154 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621175 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d8b08027-e6a6-4103-85bd-8ac9b940b103-agent-certs\") pod \"konnectivity-agent-g99r4\" (UID: \"d8b08027-e6a6-4103-85bd-8ac9b940b103\") " pod="kube-system/konnectivity-agent-g99r4" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621214 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a73de68c-3175-4736-bf34-52f955c17308-serviceca\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621231 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-lib-modules\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621250 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-netns\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-cni-bin\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621302 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-run-ovn-kubernetes\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621329 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621339 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e644e105-0aef-40d5-ac36-9ce70a2284b4-lib-modules\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-system-cni-dir\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffdqt\" (UniqueName: \"kubernetes.io/projected/fc76295c-2e54-4682-a401-5c84d779932a-kube-api-access-ffdqt\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.622495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.621687 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a73de68c-3175-4736-bf34-52f955c17308-serviceca\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.622961 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.622892 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e644e105-0aef-40d5-ac36-9ce70a2284b4-etc-tuned\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.623023 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.623007 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e644e105-0aef-40d5-ac36-9ce70a2284b4-tmp\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.631008 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.630980 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr6cv\" (UniqueName: \"kubernetes.io/projected/e644e105-0aef-40d5-ac36-9ce70a2284b4-kube-api-access-kr6cv\") pod \"tuned-z4h74\" (UID: \"e644e105-0aef-40d5-ac36-9ce70a2284b4\") " pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.631121 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.631065 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xs96\" (UniqueName: \"kubernetes.io/projected/37b8dbdd-d75f-47e5-8776-fbb29a0f2104-kube-api-access-4xs96\") pod \"node-resolver-4frtv\" (UID: \"37b8dbdd-d75f-47e5-8776-fbb29a0f2104\") " pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.632000 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.631978 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzw55\" (UniqueName: \"kubernetes.io/projected/a73de68c-3175-4736-bf34-52f955c17308-kube-api-access-jzw55\") pod \"node-ca-szzhz\" (UID: \"a73de68c-3175-4736-bf34-52f955c17308\") " pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.632089 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.632013 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l824h\" (UniqueName: \"kubernetes.io/projected/3e3418e8-e529-48ae-98b5-df0b3a3bfdbb-kube-api-access-l824h\") pod \"aws-ebs-csi-driver-node-xlbfj\" (UID: \"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.683307 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.683274 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5866c7b02ae35c0d688802a9a7d31ed6.slice/crio-ef43a299def1f2661f9cb8c127a6e358e9c2f78648098d18dfbdc56764e87ac7 WatchSource:0}: Error finding container ef43a299def1f2661f9cb8c127a6e358e9c2f78648098d18dfbdc56764e87ac7: Status 404 returned error can't find the container with id ef43a299def1f2661f9cb8c127a6e358e9c2f78648098d18dfbdc56764e87ac7 May 28 17:09:51.683658 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.683639 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14867644809a4eb2323ebb06f498f120.slice/crio-83d3e97cdc7095bd39b0844a518cefd5e28c08616a09f1afa8e3a478d852b305 WatchSource:0}: Error finding container 83d3e97cdc7095bd39b0844a518cefd5e28c08616a09f1afa8e3a478d852b305: Status 404 returned error can't find the container with id 83d3e97cdc7095bd39b0844a518cefd5e28c08616a09f1afa8e3a478d852b305 May 28 17:09:51.687773 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.687756 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 17:09:51.722008 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.721983 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-env-overrides\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722013 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-os-release\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722029 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722043 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-systemd-units\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722059 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-system-cni-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722080 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc76295c-2e54-4682-a401-5c84d779932a-cni-binary-copy\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722104 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-socket-dir-parent\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722120 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-os-release\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722136 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-system-cni-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722123 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-systemd-units\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722150 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722197 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-k8s-cni-cncf-io\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722223 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-socket-dir-parent\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722232 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-cni-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722248 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-k8s-cni-cncf-io\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722259 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-hostroot\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722292 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-conf-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722311 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-cni-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722317 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-cni-netd\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722318 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-hostroot\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-multus-conf-dir\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722355 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-node-log\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722367 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-cni-netd\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722383 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d8b08027-e6a6-4103-85bd-8ac9b940b103-konnectivity-ca\") pod \"konnectivity-agent-g99r4\" (UID: \"d8b08027-e6a6-4103-85bd-8ac9b940b103\") " pod="kube-system/konnectivity-agent-g99r4" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722408 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-node-log\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722410 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-cnibin\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722446 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-var-lib-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722451 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-cnibin\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.722640 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722473 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-kubelet\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722490 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-var-lib-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722498 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc76295c-2e54-4682-a401-5c84d779932a-multus-daemon-config\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722521 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-systemd\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722526 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-kubelet\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722547 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p8gnl\" (UniqueName: \"kubernetes.io/projected/e04539f3-03d0-41ac-8c13-0f67edc8e25c-kube-api-access-p8gnl\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722575 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-etc-kubernetes\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722579 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc76295c-2e54-4682-a401-5c84d779932a-cni-binary-copy\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722598 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-slash\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722617 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-systemd\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722624 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-log-socket\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722626 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-env-overrides\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722637 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-etc-kubernetes\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722648 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovnkube-script-lib\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722676 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722688 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-slash\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722702 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722695 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-log-socket\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.723271 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722736 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d8b08027-e6a6-4103-85bd-8ac9b940b103-agent-certs\") pod \"konnectivity-agent-g99r4\" (UID: \"d8b08027-e6a6-4103-85bd-8ac9b940b103\") " pod="kube-system/konnectivity-agent-g99r4" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.722822 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722822 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722875 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-netns\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722894 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d8b08027-e6a6-4103-85bd-8ac9b940b103-konnectivity-ca\") pod \"konnectivity-agent-g99r4\" (UID: \"d8b08027-e6a6-4103-85bd-8ac9b940b103\") " pod="kube-system/konnectivity-agent-g99r4" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722920 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-netns\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.722970 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:09:52.222891529 +0000 UTC m=+2.135284184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.722994 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-cni-bin\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723022 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-run-ovn-kubernetes\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723050 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723061 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-cni-bin\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723024 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc76295c-2e54-4682-a401-5c84d779932a-multus-daemon-config\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723104 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-system-cni-dir\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723123 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdqt\" (UniqueName: \"kubernetes.io/projected/fc76295c-2e54-4682-a401-5c84d779932a-kube-api-access-ffdqt\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723150 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-run-netns\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723178 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovn-node-metrics-cert\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723184 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-system-cni-dir\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.724127 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723205 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-cni-binary-copy\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723216 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-run-netns\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723234 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723354 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovnkube-script-lib\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723370 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-etc-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723396 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-run-ovn-kubernetes\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723410 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-cni-bin\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723443 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abe63baf-7676-4084-9037-525c520b3707-host-slash\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723492 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abe63baf-7676-4084-9037-525c520b3707-host-slash\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723509 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-etc-openvswitch\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723542 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-cni-bin\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723558 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-os-release\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723603 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723631 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-os-release\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723654 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xj9mz\" (UniqueName: \"kubernetes.io/projected/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-kube-api-access-xj9mz\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723681 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723688 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.724759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723696 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-cnibin\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723727 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7653292-2abd-4283-b387-3ad935b2be56-cnibin\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723731 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-ovn\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723756 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q5sd8\" (UniqueName: \"kubernetes.io/projected/e7653292-2abd-4283-b387-3ad935b2be56-kube-api-access-q5sd8\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723783 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-run-ovn\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723794 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abe63baf-7676-4084-9037-525c520b3707-iptables-alerter-script\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723798 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-cni-binary-copy\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723821 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bdqvg\" (UniqueName: \"kubernetes.io/projected/abe63baf-7676-4084-9037-525c520b3707-kube-api-access-bdqvg\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723849 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-cni-multus\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723873 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovnkube-config\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723902 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723910 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-var-lib-cni-multus\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.723931 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-kubelet\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.724078 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-multus-certs\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.724120 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-host-kubelet\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.724191 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc76295c-2e54-4682-a401-5c84d779932a-host-run-multus-certs\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.724372 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abe63baf-7676-4084-9037-525c520b3707-iptables-alerter-script\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.725298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.724389 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e7653292-2abd-4283-b387-3ad935b2be56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.725845 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.724468 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovnkube-config\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.725845 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.725290 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d8b08027-e6a6-4103-85bd-8ac9b940b103-agent-certs\") pod \"konnectivity-agent-g99r4\" (UID: \"d8b08027-e6a6-4103-85bd-8ac9b940b103\") " pod="kube-system/konnectivity-agent-g99r4" May 28 17:09:51.725845 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.725407 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-ovn-node-metrics-cert\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.736455 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.736421 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdqt\" (UniqueName: \"kubernetes.io/projected/fc76295c-2e54-4682-a401-5c84d779932a-kube-api-access-ffdqt\") pod \"multus-dgbcd\" (UID: \"fc76295c-2e54-4682-a401-5c84d779932a\") " pod="openshift-multus/multus-dgbcd" May 28 17:09:51.736798 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.736771 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 17:09:51.736798 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.736797 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 17:09:51.736980 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.736810 2566 projected.go:194] Error preparing data for projected volume kube-api-access-t5d8p for pod openshift-network-diagnostics/network-check-target-kxlwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:51.736980 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:51.736869 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p podName:5d76de2f-eb8b-4811-843f-8cdaa2dca0ce nodeName:}" failed. No retries permitted until 2026-05-28 17:09:52.236851886 +0000 UTC m=+2.149244530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t5d8p" (UniqueName: "kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p") pod "network-check-target-kxlwn" (UID: "5d76de2f-eb8b-4811-843f-8cdaa2dca0ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:51.737713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.737694 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5sd8\" (UniqueName: \"kubernetes.io/projected/e7653292-2abd-4283-b387-3ad935b2be56-kube-api-access-q5sd8\") pod \"multus-additional-cni-plugins-vlvsd\" (UID: \"e7653292-2abd-4283-b387-3ad935b2be56\") " pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.738515 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.738495 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdqvg\" (UniqueName: \"kubernetes.io/projected/abe63baf-7676-4084-9037-525c520b3707-kube-api-access-bdqvg\") pod \"iptables-alerter-xnpdv\" (UID: \"abe63baf-7676-4084-9037-525c520b3707\") " pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.738894 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.738875 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8gnl\" (UniqueName: \"kubernetes.io/projected/e04539f3-03d0-41ac-8c13-0f67edc8e25c-kube-api-access-p8gnl\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:51.739199 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.739185 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj9mz\" (UniqueName: \"kubernetes.io/projected/1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264-kube-api-access-xj9mz\") pod \"ovnkube-node-dnjtt\" (UID: \"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264\") " pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.841111 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.841037 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" May 28 17:09:51.846763 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.846726 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z4h74" May 28 17:09:51.847666 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.847645 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3418e8_e529_48ae_98b5_df0b3a3bfdbb.slice/crio-7a88925d524cbd6b95d42ad73a21697caf55e95f08650d9787deabdca75c7be2 WatchSource:0}: Error finding container 7a88925d524cbd6b95d42ad73a21697caf55e95f08650d9787deabdca75c7be2: Status 404 returned error can't find the container with id 7a88925d524cbd6b95d42ad73a21697caf55e95f08650d9787deabdca75c7be2 May 28 17:09:51.850099 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.850081 2566 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 28 17:09:51.852965 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.852930 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode644e105_0aef_40d5_ac36_9ce70a2284b4.slice/crio-fc65fa624aadcc2ba06f054845a1e73cb1a579402736d88cbce4010622fd4af0 WatchSource:0}: Error finding container fc65fa624aadcc2ba06f054845a1e73cb1a579402736d88cbce4010622fd4af0: Status 404 returned error can't find the container with id fc65fa624aadcc2ba06f054845a1e73cb1a579402736d88cbce4010622fd4af0 May 28 17:09:51.867053 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.867035 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4frtv" May 28 17:09:51.872422 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.872400 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b8dbdd_d75f_47e5_8776_fbb29a0f2104.slice/crio-54c664e0970294e1e7c6e37b0710066fb535c94478e16b25931904b609dc696e WatchSource:0}: Error finding container 54c664e0970294e1e7c6e37b0710066fb535c94478e16b25931904b609dc696e: Status 404 returned error can't find the container with id 54c664e0970294e1e7c6e37b0710066fb535c94478e16b25931904b609dc696e May 28 17:09:51.884226 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.884209 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-szzhz" May 28 17:09:51.890852 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.890832 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73de68c_3175_4736_bf34_52f955c17308.slice/crio-6bcc763229873ae749a240a7191a9bc4d528f3676a6795a53bdf41491370c9c3 WatchSource:0}: Error finding container 6bcc763229873ae749a240a7191a9bc4d528f3676a6795a53bdf41491370c9c3: Status 404 returned error can't find the container with id 6bcc763229873ae749a240a7191a9bc4d528f3676a6795a53bdf41491370c9c3 May 28 17:09:51.896563 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.896544 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" May 28 17:09:51.901981 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.901962 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7653292_2abd_4283_b387_3ad935b2be56.slice/crio-be7ff4f85b9026a808c3c724edd37512cbff89908d663c9b482dc6d9d613aa60 WatchSource:0}: Error finding container be7ff4f85b9026a808c3c724edd37512cbff89908d663c9b482dc6d9d613aa60: Status 404 returned error can't find the container with id be7ff4f85b9026a808c3c724edd37512cbff89908d663c9b482dc6d9d613aa60 May 28 17:09:51.910279 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.910264 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dgbcd" May 28 17:09:51.915364 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.915344 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc76295c_2e54_4682_a401_5c84d779932a.slice/crio-f5d1c179c8b2d6480f907ef4605ce48ed302a1ca6b9875f14156697e620c71e8 WatchSource:0}: Error finding container f5d1c179c8b2d6480f907ef4605ce48ed302a1ca6b9875f14156697e620c71e8: Status 404 returned error can't find the container with id f5d1c179c8b2d6480f907ef4605ce48ed302a1ca6b9875f14156697e620c71e8 May 28 17:09:51.924365 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.924345 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g99r4" May 28 17:09:51.930076 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.930050 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b08027_e6a6_4103_85bd_8ac9b940b103.slice/crio-1e6d3c6fa42469c5ca8a7b611c563cf28607df8d2cb32c0ee9591f8035bc5819 WatchSource:0}: Error finding container 1e6d3c6fa42469c5ca8a7b611c563cf28607df8d2cb32c0ee9591f8035bc5819: Status 404 returned error can't find the container with id 1e6d3c6fa42469c5ca8a7b611c563cf28607df8d2cb32c0ee9591f8035bc5819 May 28 17:09:51.949405 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.949387 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xnpdv" May 28 17:09:51.954421 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.954398 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe63baf_7676_4084_9037_525c520b3707.slice/crio-571da80d6cf75bb558e30bb90606ee7619f3346c3d009b610080127b786cb031 WatchSource:0}: Error finding container 571da80d6cf75bb558e30bb90606ee7619f3346c3d009b610080127b786cb031: Status 404 returned error can't find the container with id 571da80d6cf75bb558e30bb90606ee7619f3346c3d009b610080127b786cb031 May 28 17:09:51.955697 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:51.955685 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:09:51.961057 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:09:51.961038 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7e77b5_ba57_4a6b_bdfa_9c546c6fa264.slice/crio-ba154c0020f454e99e7f940d42730780d2020ae53ac11900bf5b745b7601a2cd WatchSource:0}: Error finding container ba154c0020f454e99e7f940d42730780d2020ae53ac11900bf5b745b7601a2cd: Status 404 returned error can't find the container with id ba154c0020f454e99e7f940d42730780d2020ae53ac11900bf5b745b7601a2cd May 28 17:09:52.228525 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.228385 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:52.228712 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:52.228533 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:52.228712 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:52.228599 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:09:53.228583511 +0000 UTC m=+3.140976141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:52.329650 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.329605 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:52.329816 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:52.329798 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 17:09:52.329885 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:52.329825 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 17:09:52.329885 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:52.329842 2566 projected.go:194] Error preparing data for projected volume kube-api-access-t5d8p for pod openshift-network-diagnostics/network-check-target-kxlwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:52.330012 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:52.329899 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p podName:5d76de2f-eb8b-4811-843f-8cdaa2dca0ce nodeName:}" failed. No retries permitted until 2026-05-28 17:09:53.329881112 +0000 UTC m=+3.242273764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5d8p" (UniqueName: "kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p") pod "network-check-target-kxlwn" (UID: "5d76de2f-eb8b-4811-843f-8cdaa2dca0ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:52.498201 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.498124 2566 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 28 17:09:52.560352 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.560256 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 17:04:51 +0000 UTC" deadline="2027-10-27 10:49:17.81319594 +0000 UTC" May 28 17:09:52.560352 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.560295 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12401h39m25.252904771s" May 28 17:09:52.665186 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.665129 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xnpdv" event={"ID":"abe63baf-7676-4084-9037-525c520b3707","Type":"ContainerStarted","Data":"571da80d6cf75bb558e30bb90606ee7619f3346c3d009b610080127b786cb031"} May 28 17:09:52.691505 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.691465 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dgbcd" event={"ID":"fc76295c-2e54-4682-a401-5c84d779932a","Type":"ContainerStarted","Data":"f5d1c179c8b2d6480f907ef4605ce48ed302a1ca6b9875f14156697e620c71e8"} May 28 17:09:52.700258 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.700213 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-szzhz" event={"ID":"a73de68c-3175-4736-bf34-52f955c17308","Type":"ContainerStarted","Data":"6bcc763229873ae749a240a7191a9bc4d528f3676a6795a53bdf41491370c9c3"} May 28 17:09:52.714051 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.714023 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4frtv" event={"ID":"37b8dbdd-d75f-47e5-8776-fbb29a0f2104","Type":"ContainerStarted","Data":"54c664e0970294e1e7c6e37b0710066fb535c94478e16b25931904b609dc696e"} May 28 17:09:52.720471 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.720443 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" event={"ID":"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb","Type":"ContainerStarted","Data":"7a88925d524cbd6b95d42ad73a21697caf55e95f08650d9787deabdca75c7be2"} May 28 17:09:52.737147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.737080 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" event={"ID":"5866c7b02ae35c0d688802a9a7d31ed6","Type":"ContainerStarted","Data":"ef43a299def1f2661f9cb8c127a6e358e9c2f78648098d18dfbdc56764e87ac7"} May 28 17:09:52.752719 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.752656 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g99r4" event={"ID":"d8b08027-e6a6-4103-85bd-8ac9b940b103","Type":"ContainerStarted","Data":"1e6d3c6fa42469c5ca8a7b611c563cf28607df8d2cb32c0ee9591f8035bc5819"} May 28 17:09:52.766893 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.766845 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerStarted","Data":"be7ff4f85b9026a808c3c724edd37512cbff89908d663c9b482dc6d9d613aa60"} May 28 17:09:52.774857 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.774830 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z4h74" event={"ID":"e644e105-0aef-40d5-ac36-9ce70a2284b4","Type":"ContainerStarted","Data":"fc65fa624aadcc2ba06f054845a1e73cb1a579402736d88cbce4010622fd4af0"} May 28 17:09:52.782305 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.782280 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" event={"ID":"14867644809a4eb2323ebb06f498f120","Type":"ContainerStarted","Data":"83d3e97cdc7095bd39b0844a518cefd5e28c08616a09f1afa8e3a478d852b305"} May 28 17:09:52.785510 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.785484 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"ba154c0020f454e99e7f940d42730780d2020ae53ac11900bf5b745b7601a2cd"} May 28 17:09:52.833685 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:52.833656 2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 17:09:53.236556 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:53.236479 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:53.236711 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:53.236659 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:53.236773 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:53.236732 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:09:55.236712842 +0000 UTC m=+5.149105479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:53.337433 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:53.337396 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:53.337609 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:53.337577 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 17:09:53.337609 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:53.337606 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 17:09:53.337718 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:53.337619 2566 projected.go:194] Error preparing data for projected volume kube-api-access-t5d8p for pod openshift-network-diagnostics/network-check-target-kxlwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:53.337718 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:53.337680 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p podName:5d76de2f-eb8b-4811-843f-8cdaa2dca0ce nodeName:}" failed. No retries permitted until 2026-05-28 17:09:55.337661181 +0000 UTC m=+5.250053813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5d8p" (UniqueName: "kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p") pod "network-check-target-kxlwn" (UID: "5d76de2f-eb8b-4811-843f-8cdaa2dca0ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:53.532482 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:53.532391 2566 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 28 17:09:53.560688 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:53.560622 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 17:04:51 +0000 UTC" deadline="2027-12-25 02:10:02.412530706 +0000 UTC" May 28 17:09:53.560688 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:53.560658 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13809h0m8.851877478s" May 28 17:09:53.637405 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:53.637372 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:53.637578 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:53.637495 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:09:53.637924 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:53.637904 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:53.638056 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:53.638026 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:09:55.256364 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:55.256325 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:55.256829 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:55.256498 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:55.256829 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:55.256574 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:09:59.256555161 +0000 UTC m=+9.168947806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:55.357818 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:55.357777 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:55.358025 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:55.357958 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 17:09:55.358025 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:55.357977 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 17:09:55.358025 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:55.357990 2566 projected.go:194] Error preparing data for projected volume kube-api-access-t5d8p for pod openshift-network-diagnostics/network-check-target-kxlwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:55.358185 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:55.358048 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p podName:5d76de2f-eb8b-4811-843f-8cdaa2dca0ce nodeName:}" failed. No retries permitted until 2026-05-28 17:09:59.358028818 +0000 UTC m=+9.270421467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5d8p" (UniqueName: "kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p") pod "network-check-target-kxlwn" (UID: "5d76de2f-eb8b-4811-843f-8cdaa2dca0ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:55.637588 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:55.637507 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:55.637733 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:55.637640 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:09:55.638347 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:55.638079 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:55.638347 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:55.638183 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:09:57.637633 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:57.637598 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:57.638118 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:57.637598 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:57.638118 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:57.637739 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:09:57.638118 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:57.637776 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:09:59.290449 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:59.289877 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:59.290449 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:59.290030 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:59.290449 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:59.290098 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:07.290077859 +0000 UTC m=+17.202470512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:09:59.390999 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:59.390959 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:59.391168 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:59.391120 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 17:09:59.391168 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:59.391151 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 17:09:59.391168 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:59.391166 2566 projected.go:194] Error preparing data for projected volume kube-api-access-t5d8p for pod openshift-network-diagnostics/network-check-target-kxlwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:59.391312 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:59.391243 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p podName:5d76de2f-eb8b-4811-843f-8cdaa2dca0ce nodeName:}" failed. No retries permitted until 2026-05-28 17:10:07.391220625 +0000 UTC m=+17.303613258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5d8p" (UniqueName: "kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p") pod "network-check-target-kxlwn" (UID: "5d76de2f-eb8b-4811-843f-8cdaa2dca0ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:09:59.637367 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:59.637246 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:09:59.637527 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:59.637364 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:09:59.637764 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:09:59.637246 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:09:59.637764 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:09:59.637725 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:01.637281 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:01.636995 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:01.637730 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:01.637330 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:01.637730 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:01.637009 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:01.637839 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:01.637805 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:03.637646 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:03.637612 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:03.637646 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:03.637635 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:03.638172 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:03.637744 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:03.638172 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:03.637877 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:05.637286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:05.637239 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:05.637286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:05.637272 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:05.637856 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:05.637381 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:05.637856 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:05.637488 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:07.350196 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:07.350157 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:07.350745 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:07.350330 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:10:07.350745 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:07.350402 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:23.350383808 +0000 UTC m=+33.262776442 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:10:07.451320 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:07.451279 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:07.451507 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:07.451446 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 17:10:07.451507 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:07.451473 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 17:10:07.451507 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:07.451484 2566 projected.go:194] Error preparing data for projected volume kube-api-access-t5d8p for pod openshift-network-diagnostics/network-check-target-kxlwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:10:07.451685 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:07.451536 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p podName:5d76de2f-eb8b-4811-843f-8cdaa2dca0ce nodeName:}" failed. No retries permitted until 2026-05-28 17:10:23.451522452 +0000 UTC m=+33.363915082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5d8p" (UniqueName: "kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p") pod "network-check-target-kxlwn" (UID: "5d76de2f-eb8b-4811-843f-8cdaa2dca0ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:10:07.636972 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:07.636880 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:07.636972 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:07.636883 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:07.637193 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:07.637017 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:07.637193 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:07.637118 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:09.637496 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:09.637411 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:09.637869 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:09.637411 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:09.637869 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:09.637586 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:09.637869 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:09.637605 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:11.637705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:11.637629 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:11.637705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:11.637645 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:11.638130 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:11.637735 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:11.638130 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:11.637878 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:11.825182 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:11.824974 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z4h74" event={"ID":"e644e105-0aef-40d5-ac36-9ce70a2284b4","Type":"ContainerStarted","Data":"1762491590d21ae3d870549fe90738430e5d7b0826a5ab8c87add923cdacd9ff"} May 28 17:10:11.830197 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:11.829964 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dgbcd" event={"ID":"fc76295c-2e54-4682-a401-5c84d779932a","Type":"ContainerStarted","Data":"72af1f91f23d53b00aad5450d144af69922c5cfe8add0fba3c24a08a5fe0fc1a"} May 28 17:10:11.846175 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:11.845725 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" event={"ID":"5866c7b02ae35c0d688802a9a7d31ed6","Type":"ContainerStarted","Data":"0a4005819eaaec43005f8a0e9f3d5fdbda3eb2575184a281d53e477791691c66"} May 28 17:10:11.858019 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:11.857970 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z4h74" podStartSLOduration=2.605083488 podStartE2EDuration="21.857935297s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.85442658 +0000 UTC m=+1.766819211" lastFinishedPulling="2026-05-28 17:10:11.107278382 +0000 UTC m=+21.019671020" observedRunningTime="2026-05-28 17:10:11.857842489 +0000 UTC m=+21.770235153" watchObservedRunningTime="2026-05-28 17:10:11.857935297 +0000 UTC m=+21.770327951" May 28 17:10:11.909330 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:11.909275 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-dgbcd" podStartSLOduration=2.580021791 podStartE2EDuration="21.90925319s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.916980444 +0000 UTC m=+1.829373075" lastFinishedPulling="2026-05-28 17:10:11.246211831 +0000 UTC m=+21.158604474" observedRunningTime="2026-05-28 17:10:11.888813132 +0000 UTC m=+21.801205992" watchObservedRunningTime="2026-05-28 17:10:11.90925319 +0000 UTC m=+21.821645844" May 28 17:10:11.909764 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:11.909727 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-46.ec2.internal" podStartSLOduration=20.90971848 podStartE2EDuration="20.90971848s" podCreationTimestamp="2026-05-28 17:09:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:10:11.909687327 +0000 UTC m=+21.822080004" watchObservedRunningTime="2026-05-28 17:10:11.90971848 +0000 UTC m=+21.822111132" May 28 17:10:12.849572 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.849317 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"0e8d64aaf9fdcb5e5c808c5f376d7609d84ca2e2ae4b8ce8c6046b87d1446af1"} May 28 17:10:12.849572 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.849531 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"ae6a22c0c8536d6aca5efd612f604465839fec9f6171f1659371e388d8d7a0ba"} May 28 17:10:12.849572 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.849552 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"d75c75beb565226d3ff8a1a3f316acc5a2aeb2db415e735f4434562cb59d9724"} May 28 17:10:12.849572 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.849566 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"59d7df61c15c2adf9cf2e66f89faf6da69c8d0813b76d7517b73d89ea04fd74d"} May 28 17:10:12.850422 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.849580 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"ca997d96ce791aa77503fbc0150ca0b6a6af2dc8ee687170b7c1fd7f4f0899ee"} May 28 17:10:12.850422 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.849589 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"83bedba7781f98cfdd8f227477f4da38aee57ba12b7b514841376de70300586b"} May 28 17:10:12.850575 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.850551 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xnpdv" event={"ID":"abe63baf-7676-4084-9037-525c520b3707","Type":"ContainerStarted","Data":"56d8e95af3450ccc66454b9ccdb0d38d57d5f1f3a887b041a426eb133e71cfdf"} May 28 17:10:12.851836 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.851814 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-szzhz" event={"ID":"a73de68c-3175-4736-bf34-52f955c17308","Type":"ContainerStarted","Data":"2339455a579e728cb7f61290546ac48d34446ae2d438876cef48b4e796027b7c"} May 28 17:10:12.852980 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.852934 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4frtv" event={"ID":"37b8dbdd-d75f-47e5-8776-fbb29a0f2104","Type":"ContainerStarted","Data":"b4d1e9860c5ddc1e697ab6ea47ba6f3a9ca241c4fc05fbd385c4abb65e757733"} May 28 17:10:12.854121 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.854104 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" event={"ID":"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb","Type":"ContainerStarted","Data":"06632bed3ffd396c6974d680aea4527980b66e7c365ce8d71b77748d6a446fbb"} May 28 17:10:12.855160 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.855136 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g99r4" event={"ID":"d8b08027-e6a6-4103-85bd-8ac9b940b103","Type":"ContainerStarted","Data":"d2f64008607e28371be2ffe945cf085a93a841cd9d629ff028eb15f2a683e528"} May 28 17:10:12.856281 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.856261 2566 generic.go:358] "Generic (PLEG): container finished" podID="e7653292-2abd-4283-b387-3ad935b2be56" containerID="9e1f2c051785011d4c6abc1a0b74e33177c22544ee5d0e28c7d72079b6868ae0" exitCode=0 May 28 17:10:12.856397 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.856313 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerDied","Data":"9e1f2c051785011d4c6abc1a0b74e33177c22544ee5d0e28c7d72079b6868ae0"} May 28 17:10:12.860026 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.858546 2566 generic.go:358] "Generic (PLEG): container finished" podID="14867644809a4eb2323ebb06f498f120" containerID="4bde15e21f23910d0399973b52791bf2e99f9cb99bd56614dbf699d09ec26abf" exitCode=0 May 28 17:10:12.860026 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.859784 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" event={"ID":"14867644809a4eb2323ebb06f498f120","Type":"ContainerDied","Data":"4bde15e21f23910d0399973b52791bf2e99f9cb99bd56614dbf699d09ec26abf"} May 28 17:10:12.879026 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.878978 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xnpdv" podStartSLOduration=3.7290796840000002 podStartE2EDuration="22.878962654s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.955720086 +0000 UTC m=+1.868112716" lastFinishedPulling="2026-05-28 17:10:11.105603056 +0000 UTC m=+21.017995686" observedRunningTime="2026-05-28 17:10:12.878907071 +0000 UTC m=+22.791299725" watchObservedRunningTime="2026-05-28 17:10:12.878962654 +0000 UTC m=+22.791355299" May 28 17:10:12.914647 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.914602 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-szzhz" podStartSLOduration=3.701124717 podStartE2EDuration="22.914580451s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.89218688 +0000 UTC m=+1.804579511" lastFinishedPulling="2026-05-28 17:10:11.105642607 +0000 UTC m=+21.018035245" observedRunningTime="2026-05-28 17:10:12.914431078 +0000 UTC m=+22.826823734" watchObservedRunningTime="2026-05-28 17:10:12.914580451 +0000 UTC m=+22.826973104" May 28 17:10:12.991592 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:12.991539 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g99r4" podStartSLOduration=3.817742614 podStartE2EDuration="22.991524322s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.931443526 +0000 UTC m=+1.843836158" lastFinishedPulling="2026-05-28 17:10:11.105225221 +0000 UTC m=+21.017617866" observedRunningTime="2026-05-28 17:10:12.991441437 +0000 UTC m=+22.903834091" watchObservedRunningTime="2026-05-28 17:10:12.991524322 +0000 UTC m=+22.903916976" May 28 17:10:13.011771 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.011719 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-4frtv" podStartSLOduration=3.780280086 podStartE2EDuration="23.011701337s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.87385918 +0000 UTC m=+1.786251811" lastFinishedPulling="2026-05-28 17:10:11.105280417 +0000 UTC m=+21.017673062" observedRunningTime="2026-05-28 17:10:13.01108696 +0000 UTC m=+22.923479612" watchObservedRunningTime="2026-05-28 17:10:13.011701337 +0000 UTC m=+22.924093990" May 28 17:10:13.394515 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.394489 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 28 17:10:13.512338 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.512305 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-j7hsv"] May 28 17:10:13.515263 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.515247 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.515322 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:13.515309 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j7hsv" podUID="5a97db59-58a0-4343-9022-a4da33d6f37e" May 28 17:10:13.586290 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.586191 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T17:10:13.394511205Z","UUID":"54b7aa8e-612f-4ada-8686-34d6dbb08235","Handler":null,"Name":"","Endpoint":""} May 28 17:10:13.589594 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.589569 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 28 17:10:13.589594 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.589599 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 28 17:10:13.591866 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.591827 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a97db59-58a0-4343-9022-a4da33d6f37e-kubelet-config\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.592079 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.591870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a97db59-58a0-4343-9022-a4da33d6f37e-dbus\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.592079 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.591904 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.637745 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.637684 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:13.637876 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.637684 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:13.637876 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:13.637779 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:13.637876 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:13.637840 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:13.692547 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.692512 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a97db59-58a0-4343-9022-a4da33d6f37e-kubelet-config\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.692705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.692553 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a97db59-58a0-4343-9022-a4da33d6f37e-dbus\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.692705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.692572 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.692705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.692635 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a97db59-58a0-4343-9022-a4da33d6f37e-kubelet-config\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.692859 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:13.692721 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 17:10:13.692859 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:13.692772 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret podName:5a97db59-58a0-4343-9022-a4da33d6f37e nodeName:}" failed. No retries permitted until 2026-05-28 17:10:14.192755311 +0000 UTC m=+24.105147943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret") pod "global-pull-secret-syncer-j7hsv" (UID: "5a97db59-58a0-4343-9022-a4da33d6f37e") : object "kube-system"/"original-pull-secret" not registered May 28 17:10:13.692859 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.692799 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a97db59-58a0-4343-9022-a4da33d6f37e-dbus\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:13.861553 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.861521 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" event={"ID":"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb","Type":"ContainerStarted","Data":"afb80e7a328f4f7bd415900aa1b59aff9e67a0df9db7c90be0be4190d2d7f133"} May 28 17:10:13.862938 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.862902 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" event={"ID":"14867644809a4eb2323ebb06f498f120","Type":"ContainerStarted","Data":"3978dfdb9046e342aae8a826ffeada6a94217c4add296f9a82240336ce28ac80"} May 28 17:10:13.878805 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:13.878764 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-46.ec2.internal" podStartSLOduration=22.878752055 podStartE2EDuration="22.878752055s" podCreationTimestamp="2026-05-28 17:09:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:10:13.878230235 +0000 UTC m=+23.790622879" watchObservedRunningTime="2026-05-28 17:10:13.878752055 +0000 UTC m=+23.791144708" May 28 17:10:14.197290 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:14.197248 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:14.197477 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:14.197414 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 17:10:14.197539 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:14.197511 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret podName:5a97db59-58a0-4343-9022-a4da33d6f37e nodeName:}" failed. No retries permitted until 2026-05-28 17:10:15.197490333 +0000 UTC m=+25.109882979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret") pod "global-pull-secret-syncer-j7hsv" (UID: "5a97db59-58a0-4343-9022-a4da33d6f37e") : object "kube-system"/"original-pull-secret" not registered May 28 17:10:14.867964 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:14.867850 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"ab9c577f285f3aeb64ed54c94fbbb693ca4b281d9da996486e50d2ea2d7b211b"} May 28 17:10:14.869732 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:14.869704 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" event={"ID":"3e3418e8-e529-48ae-98b5-df0b3a3bfdbb","Type":"ContainerStarted","Data":"d9887a793e0cec0e30956f6c47e0d8775d901e2738ece10138f7b2099416fa99"} May 28 17:10:14.892568 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:14.892526 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xlbfj" podStartSLOduration=2.148131401 podStartE2EDuration="24.892512883s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.849848235 +0000 UTC m=+1.762240866" lastFinishedPulling="2026-05-28 17:10:14.594229714 +0000 UTC m=+24.506622348" observedRunningTime="2026-05-28 17:10:14.892237594 +0000 UTC m=+24.804630247" watchObservedRunningTime="2026-05-28 17:10:14.892512883 +0000 UTC m=+24.804905536" May 28 17:10:15.204286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:15.204207 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:15.204429 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:15.204362 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 17:10:15.204479 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:15.204438 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret podName:5a97db59-58a0-4343-9022-a4da33d6f37e nodeName:}" failed. No retries permitted until 2026-05-28 17:10:17.204419743 +0000 UTC m=+27.116812383 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret") pod "global-pull-secret-syncer-j7hsv" (UID: "5a97db59-58a0-4343-9022-a4da33d6f37e") : object "kube-system"/"original-pull-secret" not registered May 28 17:10:15.605313 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:15.605277 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g99r4" May 28 17:10:15.605885 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:15.605865 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g99r4" May 28 17:10:15.637209 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:15.637183 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:15.637362 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:15.637183 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:15.637362 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:15.637297 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j7hsv" podUID="5a97db59-58a0-4343-9022-a4da33d6f37e" May 28 17:10:15.637362 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:15.637191 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:15.637511 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:15.637367 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:15.637511 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:15.637431 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:15.871812 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:15.871723 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g99r4" May 28 17:10:15.872417 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:15.872180 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g99r4" May 28 17:10:16.877167 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:16.877126 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" event={"ID":"1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264","Type":"ContainerStarted","Data":"6532600e2d6653390bda72cf129175d3547b14bebf4c22ecbbeaee00e88e1f21"} May 28 17:10:17.221624 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.221590 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:17.221785 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:17.221726 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 17:10:17.221785 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:17.221781 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret podName:5a97db59-58a0-4343-9022-a4da33d6f37e nodeName:}" failed. No retries permitted until 2026-05-28 17:10:21.221768438 +0000 UTC m=+31.134161074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret") pod "global-pull-secret-syncer-j7hsv" (UID: "5a97db59-58a0-4343-9022-a4da33d6f37e") : object "kube-system"/"original-pull-secret" not registered May 28 17:10:17.637524 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.637493 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:17.637663 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.637494 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:17.637663 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:17.637600 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j7hsv" podUID="5a97db59-58a0-4343-9022-a4da33d6f37e" May 28 17:10:17.637779 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.637494 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:17.637779 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:17.637678 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:17.637874 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:17.637794 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:17.882138 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.879822 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:10:17.882138 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.879862 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:10:17.882138 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.879875 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:10:17.901582 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.901484 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:10:17.902719 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.902687 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:10:17.936266 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:17.936119 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" podStartSLOduration=8.114134443 podStartE2EDuration="27.936104071s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.962370598 +0000 UTC m=+1.874763229" lastFinishedPulling="2026-05-28 17:10:11.784340223 +0000 UTC m=+21.696732857" observedRunningTime="2026-05-28 17:10:16.916014562 +0000 UTC m=+26.828407216" watchObservedRunningTime="2026-05-28 17:10:17.936104071 +0000 UTC m=+27.848496725" May 28 17:10:18.919919 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:18.919703 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j7hsv"] May 28 17:10:18.920414 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:18.920054 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:18.920414 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:18.920150 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j7hsv" podUID="5a97db59-58a0-4343-9022-a4da33d6f37e" May 28 17:10:18.922481 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:18.922451 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kxlwn"] May 28 17:10:18.922591 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:18.922561 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:18.922674 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:18.922647 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:18.923176 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:18.923155 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lsl89"] May 28 17:10:18.923285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:18.923269 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:18.923397 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:18.923376 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:19.883928 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:19.883884 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerStarted","Data":"f7d2dbdc8eef336711eb4869c0a754c7b303d820295f2360008852ebb506742a"} May 28 17:10:20.638366 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:20.638337 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:20.638691 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:20.638434 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:20.638691 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:20.638523 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:20.638691 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:20.638628 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:20.638691 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:20.638677 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:20.638822 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:20.638749 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j7hsv" podUID="5a97db59-58a0-4343-9022-a4da33d6f37e" May 28 17:10:20.887136 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:20.887077 2566 generic.go:358] "Generic (PLEG): container finished" podID="e7653292-2abd-4283-b387-3ad935b2be56" containerID="f7d2dbdc8eef336711eb4869c0a754c7b303d820295f2360008852ebb506742a" exitCode=0 May 28 17:10:20.887282 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:20.887157 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerDied","Data":"f7d2dbdc8eef336711eb4869c0a754c7b303d820295f2360008852ebb506742a"} May 28 17:10:21.257492 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:21.257459 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:21.257681 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:21.257614 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 17:10:21.257749 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:21.257706 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret podName:5a97db59-58a0-4343-9022-a4da33d6f37e nodeName:}" failed. No retries permitted until 2026-05-28 17:10:29.257682161 +0000 UTC m=+39.170074792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret") pod "global-pull-secret-syncer-j7hsv" (UID: "5a97db59-58a0-4343-9022-a4da33d6f37e") : object "kube-system"/"original-pull-secret" not registered May 28 17:10:22.637250 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:22.637216 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:22.637250 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:22.637231 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:22.637805 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:22.637322 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j7hsv" podUID="5a97db59-58a0-4343-9022-a4da33d6f37e" May 28 17:10:22.637805 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:22.637373 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:10:22.637805 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:22.637420 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:22.637805 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:22.637497 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxlwn" podUID="5d76de2f-eb8b-4811-843f-8cdaa2dca0ce" May 28 17:10:22.892563 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:22.892480 2566 generic.go:358] "Generic (PLEG): container finished" podID="e7653292-2abd-4283-b387-3ad935b2be56" containerID="f678fdfa1212a1eb7e3664521149f2d0eddcbab0adaaf3d7aabe530c218793fd" exitCode=0 May 28 17:10:22.892563 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:22.892532 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerDied","Data":"f678fdfa1212a1eb7e3664521149f2d0eddcbab0adaaf3d7aabe530c218793fd"} May 28 17:10:23.375893 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:23.375858 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:23.376082 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:23.376054 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:10:23.376149 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:23.376127 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:55.376107933 +0000 UTC m=+65.288500571 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 17:10:23.476412 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:23.476382 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:23.476541 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:23.476493 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 17:10:23.476541 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:23.476505 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 17:10:23.476541 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:23.476514 2566 projected.go:194] Error preparing data for projected volume kube-api-access-t5d8p for pod openshift-network-diagnostics/network-check-target-kxlwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:10:23.476624 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:23.476557 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p podName:5d76de2f-eb8b-4811-843f-8cdaa2dca0ce nodeName:}" failed. No retries permitted until 2026-05-28 17:10:55.476545536 +0000 UTC m=+65.388938167 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5d8p" (UniqueName: "kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p") pod "network-check-target-kxlwn" (UID: "5d76de2f-eb8b-4811-843f-8cdaa2dca0ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 17:10:24.377622 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.377449 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-46.ec2.internal" event="NodeReady" May 28 17:10:24.378172 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.377721 2566 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 28 17:10:24.440041 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.440013 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rqq5s"] May 28 17:10:24.452868 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.452846 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h68jj"] May 28 17:10:24.453031 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.453008 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:24.460472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.459409 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 28 17:10:24.460472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.459652 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jgwcl\"" May 28 17:10:24.460472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.459671 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 28 17:10:24.460472 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.459678 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 28 17:10:24.465157 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.465137 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rqq5s"] May 28 17:10:24.465157 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.465159 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h68jj"] May 28 17:10:24.465282 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.465241 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.471444 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.471423 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 28 17:10:24.472087 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.472062 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 28 17:10:24.472189 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.472123 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmnj9\"" May 28 17:10:24.585986 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.585933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqtrt\" (UniqueName: \"kubernetes.io/projected/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-kube-api-access-qqtrt\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.586201 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.585997 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-config-volume\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.586201 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.586062 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:24.586201 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.586084 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.586201 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.586133 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-tmp-dir\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.586201 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.586188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qq75\" (UniqueName: \"kubernetes.io/projected/9e5b6585-37d1-4e39-8599-509b085c834a-kube-api-access-8qq75\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:24.637210 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.637151 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:24.637210 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.637151 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:24.637409 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.637151 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:24.640953 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.640924 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 28 17:10:24.641077 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.641059 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 17:10:24.641077 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.641072 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 17:10:24.641181 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.641144 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t7ndh\"" May 28 17:10:24.641220 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.641206 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hshhh\"" May 28 17:10:24.641443 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.641428 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 17:10:24.686474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.686453 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:24.686573 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.686479 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.686573 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.686498 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-tmp-dir\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.686641 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:24.686588 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:10:24.686641 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.686614 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8qq75\" (UniqueName: \"kubernetes.io/projected/9e5b6585-37d1-4e39-8599-509b085c834a-kube-api-access-8qq75\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:24.686641 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:24.686640 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:10:25.186626031 +0000 UTC m=+35.099018662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:10:24.686753 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:24.686593 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:10:24.686753 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.686690 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qqtrt\" (UniqueName: \"kubernetes.io/projected/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-kube-api-access-qqtrt\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.686753 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:24.686705 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:25.18669042 +0000 UTC m=+35.099083066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:10:24.686753 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.686727 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-tmp-dir\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.686753 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.686743 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-config-volume\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.687219 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.687202 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-config-volume\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.703649 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.703630 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqtrt\" (UniqueName: \"kubernetes.io/projected/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-kube-api-access-qqtrt\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:24.703765 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.703682 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qq75\" (UniqueName: \"kubernetes.io/projected/9e5b6585-37d1-4e39-8599-509b085c834a-kube-api-access-8qq75\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:24.898154 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.898075 2566 generic.go:358] "Generic (PLEG): container finished" podID="e7653292-2abd-4283-b387-3ad935b2be56" containerID="c2f366db28fe4872b49d0fd3261bd61f535d66ac4c2011719ec9ea4c1c995fc0" exitCode=0 May 28 17:10:24.898154 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:24.898134 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerDied","Data":"c2f366db28fe4872b49d0fd3261bd61f535d66ac4c2011719ec9ea4c1c995fc0"} May 28 17:10:25.192076 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:25.191999 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:25.192076 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:25.192032 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:25.192245 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:25.192127 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:10:25.192245 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:25.192162 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:10:25.192245 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:25.192181 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:10:26.192166723 +0000 UTC m=+36.104559354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:10:25.192245 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:25.192203 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:26.192192041 +0000 UTC m=+36.104584671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:10:26.200236 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:26.200200 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:26.200236 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:26.200240 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:26.200691 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:26.200333 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:10:26.200691 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:26.200364 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:10:26.200691 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:26.200393 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:28.200375565 +0000 UTC m=+38.112768196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:10:26.200691 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:26.200425 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:10:28.200407419 +0000 UTC m=+38.112800062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:10:28.216481 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:28.216430 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:28.217120 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:28.216492 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:28.217120 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:28.216578 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:10:28.217120 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:28.216645 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:10:28.217120 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:28.216654 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:10:32.216635611 +0000 UTC m=+42.129028248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:10:28.217120 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:28.216715 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:32.216698087 +0000 UTC m=+42.129090717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:10:29.325959 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:29.325914 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:29.328720 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:29.328697 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a97db59-58a0-4343-9022-a4da33d6f37e-original-pull-secret\") pod \"global-pull-secret-syncer-j7hsv\" (UID: \"5a97db59-58a0-4343-9022-a4da33d6f37e\") " pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:29.451884 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:29.451843 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j7hsv" May 28 17:10:31.166897 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:31.166832 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j7hsv"] May 28 17:10:31.230634 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:10:31.230603 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a97db59_58a0_4343_9022_a4da33d6f37e.slice/crio-efac49e97478ec23745d8ceb3fefb565d1d37e2c766cb06536f1e5b6d3e94ef7 WatchSource:0}: Error finding container efac49e97478ec23745d8ceb3fefb565d1d37e2c766cb06536f1e5b6d3e94ef7: Status 404 returned error can't find the container with id efac49e97478ec23745d8ceb3fefb565d1d37e2c766cb06536f1e5b6d3e94ef7 May 28 17:10:31.915558 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:31.915518 2566 generic.go:358] "Generic (PLEG): container finished" podID="e7653292-2abd-4283-b387-3ad935b2be56" containerID="53211deaf68f95d5560eb39cca714ab41b82dd0c3db71357118c132febc2c880" exitCode=0 May 28 17:10:31.915715 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:31.915612 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerDied","Data":"53211deaf68f95d5560eb39cca714ab41b82dd0c3db71357118c132febc2c880"} May 28 17:10:31.916742 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:31.916722 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j7hsv" event={"ID":"5a97db59-58a0-4343-9022-a4da33d6f37e","Type":"ContainerStarted","Data":"efac49e97478ec23745d8ceb3fefb565d1d37e2c766cb06536f1e5b6d3e94ef7"} May 28 17:10:32.246225 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:32.246189 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:32.246642 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:32.246232 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:32.246642 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:32.246322 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:10:32.246642 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:32.246326 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:10:32.246642 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:32.246383 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:40.24636434 +0000 UTC m=+50.158756975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:10:32.246642 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:32.246397 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:10:40.246390687 +0000 UTC m=+50.158783318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:10:32.921188 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:32.921155 2566 generic.go:358] "Generic (PLEG): container finished" podID="e7653292-2abd-4283-b387-3ad935b2be56" containerID="ea829996d5a1aac82769a29a05453a1f42f6d034a9ba7fda0a759643b8a00995" exitCode=0 May 28 17:10:32.921352 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:32.921199 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerDied","Data":"ea829996d5a1aac82769a29a05453a1f42f6d034a9ba7fda0a759643b8a00995"} May 28 17:10:33.926652 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:33.926619 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" event={"ID":"e7653292-2abd-4283-b387-3ad935b2be56","Type":"ContainerStarted","Data":"8937e34a33bccbcb1b09b27b37457a9208e36b4c84267b0cb485c85a5a51f7d4"} May 28 17:10:35.932034 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:35.931997 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j7hsv" event={"ID":"5a97db59-58a0-4343-9022-a4da33d6f37e","Type":"ContainerStarted","Data":"90bc55ef229663b1fa1bb3d9eec71ce39410decb9ea012ae9d7c7150ea44e44d"} May 28 17:10:35.955659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:35.955608 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vlvsd" podStartSLOduration=6.5990765190000005 podStartE2EDuration="45.955594262s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:09:51.903331707 +0000 UTC m=+1.815724339" lastFinishedPulling="2026-05-28 17:10:31.259849451 +0000 UTC m=+41.172242082" observedRunningTime="2026-05-28 17:10:33.957396805 +0000 UTC m=+43.869789457" watchObservedRunningTime="2026-05-28 17:10:35.955594262 +0000 UTC m=+45.867986916" May 28 17:10:35.956010 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:35.955984 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j7hsv" podStartSLOduration=19.010361652 podStartE2EDuration="22.955977147s" podCreationTimestamp="2026-05-28 17:10:13 +0000 UTC" firstStartedPulling="2026-05-28 17:10:31.238156533 +0000 UTC m=+41.150549171" lastFinishedPulling="2026-05-28 17:10:35.183772031 +0000 UTC m=+45.096164666" observedRunningTime="2026-05-28 17:10:35.955375256 +0000 UTC m=+45.867767910" watchObservedRunningTime="2026-05-28 17:10:35.955977147 +0000 UTC m=+45.868369802" May 28 17:10:40.303635 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:40.303576 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:40.303635 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:40.303639 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:40.304194 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:40.303713 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:10:40.304194 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:40.303781 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:10:56.303766589 +0000 UTC m=+66.216159221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:10:40.304194 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:40.303780 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:10:40.304194 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:40.303806 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:56.303800088 +0000 UTC m=+66.216192719 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:10:49.894416 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:49.894384 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dnjtt" May 28 17:10:55.406647 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.406595 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:10:55.409574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.409555 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 17:10:55.417793 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:55.417774 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 17:10:55.417864 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:55.417830 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:11:59.41781556 +0000 UTC m=+129.330208191 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : secret "metrics-daemon-secret" not found May 28 17:10:55.507520 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.507490 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:55.510812 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.510794 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 17:10:55.520927 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.520909 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 17:10:55.531539 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.531511 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5d8p\" (UniqueName: \"kubernetes.io/projected/5d76de2f-eb8b-4811-843f-8cdaa2dca0ce-kube-api-access-t5d8p\") pod \"network-check-target-kxlwn\" (UID: \"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce\") " pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:55.561483 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.561455 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hshhh\"" May 28 17:10:55.568982 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.568963 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:55.701551 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.701482 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kxlwn"] May 28 17:10:55.704617 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:10:55.704587 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d76de2f_eb8b_4811_843f_8cdaa2dca0ce.slice/crio-5e7ecb7b8852edebe9009220930731616623a7fca9348cdea9265a116851f858 WatchSource:0}: Error finding container 5e7ecb7b8852edebe9009220930731616623a7fca9348cdea9265a116851f858: Status 404 returned error can't find the container with id 5e7ecb7b8852edebe9009220930731616623a7fca9348cdea9265a116851f858 May 28 17:10:55.972023 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:55.971922 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kxlwn" event={"ID":"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce","Type":"ContainerStarted","Data":"5e7ecb7b8852edebe9009220930731616623a7fca9348cdea9265a116851f858"} May 28 17:10:56.313039 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:56.312999 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:10:56.313039 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:56.313037 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:10:56.313241 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:56.313140 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:10:56.313241 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:56.313141 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:10:56.313241 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:56.313195 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:11:28.31318004 +0000 UTC m=+98.225572671 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:10:56.313241 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:10:56.313207 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:11:28.313201328 +0000 UTC m=+98.225593959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:10:58.979361 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:58.979330 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kxlwn" event={"ID":"5d76de2f-eb8b-4811-843f-8cdaa2dca0ce","Type":"ContainerStarted","Data":"fd54a95d7c9c88c2dc6cd0822bf0852e2d2f56c86a2b034a94eca12cd72b1821"} May 28 17:10:58.979763 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:58.979466 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:10:58.998730 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:10:58.998684 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kxlwn" podStartSLOduration=65.944335048 podStartE2EDuration="1m8.998668212s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:10:55.706467963 +0000 UTC m=+65.618860597" lastFinishedPulling="2026-05-28 17:10:58.760801127 +0000 UTC m=+68.673193761" observedRunningTime="2026-05-28 17:10:58.998334114 +0000 UTC m=+68.910726768" watchObservedRunningTime="2026-05-28 17:10:58.998668212 +0000 UTC m=+68.911060863" May 28 17:11:28.327826 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:11:28.327775 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:11:28.327826 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:11:28.327830 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:11:28.328416 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:11:28.327924 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:11:28.328416 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:11:28.327923 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:11:28.328416 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:11:28.328029 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:12:32.328008114 +0000 UTC m=+162.240400765 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:11:28.328416 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:11:28.328087 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:12:32.328074315 +0000 UTC m=+162.240466947 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:11:29.983901 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:11:29.983869 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kxlwn" May 28 17:11:59.439864 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:11:59.439798 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:11:59.440365 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:11:59.439972 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 17:11:59.440365 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:11:59.440039 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs podName:e04539f3-03d0-41ac-8c13-0f67edc8e25c nodeName:}" failed. No retries permitted until 2026-05-28 17:14:01.440021191 +0000 UTC m=+251.352413822 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs") pod "network-metrics-daemon-lsl89" (UID: "e04539f3-03d0-41ac-8c13-0f67edc8e25c") : secret "metrics-daemon-secret" not found May 28 17:12:22.548819 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.548783 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p"] May 28 17:12:22.551737 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.551716 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p" May 28 17:12:22.558548 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.558531 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" May 28 17:12:22.559432 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.559412 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-46bgf\"" May 28 17:12:22.559508 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.559487 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" May 28 17:12:22.568855 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.568834 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p"] May 28 17:12:22.641395 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.641365 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq"] May 28 17:12:22.644104 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.644089 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq" May 28 17:12:22.649683 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.649541 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-x9pc5\"" May 28 17:12:22.650198 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.650178 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d5f85c4d6-gs4zx"] May 28 17:12:22.652894 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.652878 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.658406 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.658391 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 28 17:12:22.661703 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.661689 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zvwff\"" May 28 17:12:22.661759 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.661715 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 28 17:12:22.661792 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.661774 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 28 17:12:22.661981 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.661939 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 28 17:12:22.662052 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.662039 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 28 17:12:22.663067 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.663055 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 28 17:12:22.675101 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.675080 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq"] May 28 17:12:22.677843 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.677825 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d5f85c4d6-gs4zx"] May 28 17:12:22.691373 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.691339 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h55x\" (UniqueName: \"kubernetes.io/projected/b0750930-f62e-4151-83ec-ee92469cfa02-kube-api-access-4h55x\") pod \"volume-data-source-validator-5cf8c89d64-spj4p\" (UID: \"b0750930-f62e-4151-83ec-ee92469cfa02\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p" May 28 17:12:22.757242 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.757206 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf"] May 28 17:12:22.760026 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.760007 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:22.764921 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.764902 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" May 28 17:12:22.766422 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.766403 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" May 28 17:12:22.766509 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.766495 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" May 28 17:12:22.766914 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.766898 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" May 28 17:12:22.767088 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.767072 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-942gt\"" May 28 17:12:22.768487 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.768469 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9"] May 28 17:12:22.771145 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.771128 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.774603 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.774585 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" May 28 17:12:22.774710 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.774694 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" May 28 17:12:22.775508 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.775491 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" May 28 17:12:22.775594 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.775534 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" May 28 17:12:22.784274 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.784248 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b796f6cf6-n4trr"] May 28 17:12:22.786860 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.786847 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.787850 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.787825 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-chjhz\"" May 28 17:12:22.790309 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.790294 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 28 17:12:22.790432 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.790404 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zjnfc\"" May 28 17:12:22.790701 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.790355 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 28 17:12:22.790775 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.790301 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 28 17:12:22.791718 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.791698 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbs6x\" (UniqueName: \"kubernetes.io/projected/f8bee3dd-7589-414a-b93a-816a92414e88-kube-api-access-nbs6x\") pod \"network-check-source-7f44b6dd87-r8pdq\" (UID: \"f8bee3dd-7589-414a-b93a-816a92414e88\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq" May 28 17:12:22.791807 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.791733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.791807 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.791768 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.791912 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.791833 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw8w2\" (UniqueName: \"kubernetes.io/projected/ca21983b-3973-4275-86f0-65d6fb0410f1-kube-api-access-kw8w2\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.791912 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.791864 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4h55x\" (UniqueName: \"kubernetes.io/projected/b0750930-f62e-4151-83ec-ee92469cfa02-kube-api-access-4h55x\") pod \"volume-data-source-validator-5cf8c89d64-spj4p\" (UID: \"b0750930-f62e-4151-83ec-ee92469cfa02\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p" May 28 17:12:22.791912 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.791886 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-stats-auth\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.792071 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.791915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-default-certificate\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.798564 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.798540 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf"] May 28 17:12:22.806541 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.806477 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9"] May 28 17:12:22.809256 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.808887 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 28 17:12:22.814575 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.814557 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b796f6cf6-n4trr"] May 28 17:12:22.831692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.831673 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h55x\" (UniqueName: \"kubernetes.io/projected/b0750930-f62e-4151-83ec-ee92469cfa02-kube-api-access-4h55x\") pod \"volume-data-source-validator-5cf8c89d64-spj4p\" (UID: \"b0750930-f62e-4151-83ec-ee92469cfa02\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p" May 28 17:12:22.859314 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.859293 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p" May 28 17:12:22.892885 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.892852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-config\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.892885 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.892886 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.893074 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.892909 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncm99\" (UniqueName: \"kubernetes.io/projected/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-kube-api-access-ncm99\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.893074 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.892928 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.893074 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893023 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kw8w2\" (UniqueName: \"kubernetes.io/projected/ca21983b-3973-4275-86f0-65d6fb0410f1-kube-api-access-kw8w2\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.893074 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:22.893052 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:23.393038253 +0000 UTC m=+153.305430884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : configmap references non-existent config key: service-ca.crt May 28 17:12:22.893074 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:22.893053 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 17:12:22.893286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893078 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/907ac90b-550c-4975-b881-6e7398ee2437-serving-cert\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:22.893286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4nbs\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-kube-api-access-w4nbs\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.893286 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:22.893119 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:23.393100309 +0000 UTC m=+153.305492964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : secret "router-metrics-certs-default" not found May 28 17:12:22.893286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893178 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-default-certificate\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.893286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-image-registry-private-configuration\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.893286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893282 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nbs6x\" (UniqueName: \"kubernetes.io/projected/f8bee3dd-7589-414a-b93a-816a92414e88-kube-api-access-nbs6x\") pod \"network-check-source-7f44b6dd87-r8pdq\" (UID: \"f8bee3dd-7589-414a-b93a-816a92414e88\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893312 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-trusted-ca\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-installation-pull-secrets\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8nrr\" (UniqueName: \"kubernetes.io/projected/907ac90b-550c-4975-b881-6e7398ee2437-kube-api-access-d8nrr\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893420 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86aa82f8-5676-47b2-8cb2-03731a76fb6a-ca-trust-extracted\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893452 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/907ac90b-550c-4975-b881-6e7398ee2437-config\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893484 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-stats-auth\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893509 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893578 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-bound-sa-token\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.893659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.893620 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-certificates\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.896244 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.896216 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-default-certificate\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.897034 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.897014 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-stats-auth\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.907083 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.906332 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbs6x\" (UniqueName: \"kubernetes.io/projected/f8bee3dd-7589-414a-b93a-816a92414e88-kube-api-access-nbs6x\") pod \"network-check-source-7f44b6dd87-r8pdq\" (UID: \"f8bee3dd-7589-414a-b93a-816a92414e88\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq" May 28 17:12:22.907365 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.907340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw8w2\" (UniqueName: \"kubernetes.io/projected/ca21983b-3973-4275-86f0-65d6fb0410f1-kube-api-access-kw8w2\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:22.952442 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.952415 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq" May 28 17:12:22.983022 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.982991 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p"] May 28 17:12:22.986716 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:12:22.986685 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0750930_f62e_4151_83ec_ee92469cfa02.slice/crio-8182553befed54780ea441aa5070204d15fec4f77ce0fa949198324c6c40f52f WatchSource:0}: Error finding container 8182553befed54780ea441aa5070204d15fec4f77ce0fa949198324c6c40f52f: Status 404 returned error can't find the container with id 8182553befed54780ea441aa5070204d15fec4f77ce0fa949198324c6c40f52f May 28 17:12:22.994491 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994466 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-image-registry-private-configuration\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.994588 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994522 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-trusted-ca\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.994588 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994554 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-installation-pull-secrets\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.994588 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994583 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d8nrr\" (UniqueName: \"kubernetes.io/projected/907ac90b-550c-4975-b881-6e7398ee2437-kube-api-access-d8nrr\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:22.994750 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994614 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86aa82f8-5676-47b2-8cb2-03731a76fb6a-ca-trust-extracted\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.994750 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994642 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/907ac90b-550c-4975-b881-6e7398ee2437-config\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:22.994750 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994678 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.994750 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994705 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.994750 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994733 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-bound-sa-token\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.995017 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994764 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-certificates\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.995017 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994810 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-config\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.995017 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994851 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ncm99\" (UniqueName: \"kubernetes.io/projected/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-kube-api-access-ncm99\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.995017 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994909 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/907ac90b-550c-4975-b881-6e7398ee2437-serving-cert\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:22.995017 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.994936 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w4nbs\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-kube-api-access-w4nbs\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.995246 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.995079 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86aa82f8-5676-47b2-8cb2-03731a76fb6a-ca-trust-extracted\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.996153 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:22.995317 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 17:12:22.996153 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:22.995337 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b796f6cf6-n4trr: secret "image-registry-tls" not found May 28 17:12:22.996153 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:22.995387 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls podName:86aa82f8-5676-47b2-8cb2-03731a76fb6a nodeName:}" failed. No retries permitted until 2026-05-28 17:12:23.49536823 +0000 UTC m=+153.407760865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls") pod "image-registry-b796f6cf6-n4trr" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a") : secret "image-registry-tls" not found May 28 17:12:22.996153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.995316 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/907ac90b-550c-4975-b881-6e7398ee2437-config\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:22.996153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.995738 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-trusted-ca\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.996153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.995899 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-certificates\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.996153 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.996109 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-config\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.997574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.997548 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-installation-pull-secrets\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.997975 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.997668 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:22.997975 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.997875 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-image-registry-private-configuration\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:22.998211 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:22.998173 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/907ac90b-550c-4975-b881-6e7398ee2437-serving-cert\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:23.005815 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.005771 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8nrr\" (UniqueName: \"kubernetes.io/projected/907ac90b-550c-4975-b881-6e7398ee2437-kube-api-access-d8nrr\") pod \"service-ca-operator-57b5b68b8-82mkf\" (UID: \"907ac90b-550c-4975-b881-6e7398ee2437\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:23.006458 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.006410 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4nbs\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-kube-api-access-w4nbs\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:23.007471 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.007109 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-bound-sa-token\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:23.007975 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.007907 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncm99\" (UniqueName: \"kubernetes.io/projected/b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d-kube-api-access-ncm99\") pod \"kube-storage-version-migrator-operator-565644f49-nm9j9\" (UID: \"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:23.068431 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.068367 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" May 28 17:12:23.074437 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.074415 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq"] May 28 17:12:23.077351 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:12:23.077322 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8bee3dd_7589_414a_b93a_816a92414e88.slice/crio-a8e471d87e6192e0c74befa8055daf51a7485ccfecdd363b761ab230621506de WatchSource:0}: Error finding container a8e471d87e6192e0c74befa8055daf51a7485ccfecdd363b761ab230621506de: Status 404 returned error can't find the container with id a8e471d87e6192e0c74befa8055daf51a7485ccfecdd363b761ab230621506de May 28 17:12:23.078424 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.078403 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" May 28 17:12:23.137039 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.136994 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p" event={"ID":"b0750930-f62e-4151-83ec-ee92469cfa02","Type":"ContainerStarted","Data":"8182553befed54780ea441aa5070204d15fec4f77ce0fa949198324c6c40f52f"} May 28 17:12:23.139687 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.139113 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq" event={"ID":"f8bee3dd-7589-414a-b93a-816a92414e88","Type":"ContainerStarted","Data":"a8e471d87e6192e0c74befa8055daf51a7485ccfecdd363b761ab230621506de"} May 28 17:12:23.201268 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.201235 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf"] May 28 17:12:23.204495 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:12:23.204464 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod907ac90b_550c_4975_b881_6e7398ee2437.slice/crio-ffaa24841d1f69c85a1cb03e759e9aa36bba1bafdeff43704235b9822b0a2f5f WatchSource:0}: Error finding container ffaa24841d1f69c85a1cb03e759e9aa36bba1bafdeff43704235b9822b0a2f5f: Status 404 returned error can't find the container with id ffaa24841d1f69c85a1cb03e759e9aa36bba1bafdeff43704235b9822b0a2f5f May 28 17:12:23.212826 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.212803 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9"] May 28 17:12:23.215588 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:12:23.215565 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9bf3aa0_0375_4ecb_afe6_51a56dd8d31d.slice/crio-e7ce452649fa61c652ef6950c693a3915e416f3a274a2d7bab41dbd5616e839e WatchSource:0}: Error finding container e7ce452649fa61c652ef6950c693a3915e416f3a274a2d7bab41dbd5616e839e: Status 404 returned error can't find the container with id e7ce452649fa61c652ef6950c693a3915e416f3a274a2d7bab41dbd5616e839e May 28 17:12:23.398570 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.398483 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:23.398570 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.398515 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:23.398789 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:23.398638 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 17:12:23.398789 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:23.398652 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:24.398636799 +0000 UTC m=+154.311029430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : configmap references non-existent config key: service-ca.crt May 28 17:12:23.398789 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:23.398712 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:24.39869456 +0000 UTC m=+154.311087191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : secret "router-metrics-certs-default" not found May 28 17:12:23.499009 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:23.498971 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:23.499209 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:23.499137 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 17:12:23.499209 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:23.499160 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b796f6cf6-n4trr: secret "image-registry-tls" not found May 28 17:12:23.499314 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:23.499220 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls podName:86aa82f8-5676-47b2-8cb2-03731a76fb6a nodeName:}" failed. No retries permitted until 2026-05-28 17:12:24.499205054 +0000 UTC m=+154.411597688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls") pod "image-registry-b796f6cf6-n4trr" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a") : secret "image-registry-tls" not found May 28 17:12:24.145558 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:24.145489 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq" event={"ID":"f8bee3dd-7589-414a-b93a-816a92414e88","Type":"ContainerStarted","Data":"e0c5c13438e143e87a46ecb817c2ce480db0591364f4bade9380f025e5dad1d5"} May 28 17:12:24.147202 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:24.147169 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" event={"ID":"907ac90b-550c-4975-b881-6e7398ee2437","Type":"ContainerStarted","Data":"ffaa24841d1f69c85a1cb03e759e9aa36bba1bafdeff43704235b9822b0a2f5f"} May 28 17:12:24.148575 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:24.148547 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" event={"ID":"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d","Type":"ContainerStarted","Data":"e7ce452649fa61c652ef6950c693a3915e416f3a274a2d7bab41dbd5616e839e"} May 28 17:12:24.163214 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:24.163171 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-r8pdq" podStartSLOduration=2.163148015 podStartE2EDuration="2.163148015s" podCreationTimestamp="2026-05-28 17:12:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:12:24.163041713 +0000 UTC m=+154.075434367" watchObservedRunningTime="2026-05-28 17:12:24.163148015 +0000 UTC m=+154.075540661" May 28 17:12:24.407279 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:24.407179 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:24.407279 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:24.407242 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:24.407517 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:24.407391 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 17:12:24.407517 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:24.407486 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:26.407462309 +0000 UTC m=+156.319854962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : secret "router-metrics-certs-default" not found May 28 17:12:24.408151 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:24.408123 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:26.40807687 +0000 UTC m=+156.320469517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : configmap references non-existent config key: service-ca.crt May 28 17:12:24.508163 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:24.508099 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:24.508334 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:24.508262 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 17:12:24.508334 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:24.508284 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b796f6cf6-n4trr: secret "image-registry-tls" not found May 28 17:12:24.508441 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:24.508345 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls podName:86aa82f8-5676-47b2-8cb2-03731a76fb6a nodeName:}" failed. No retries permitted until 2026-05-28 17:12:26.508325912 +0000 UTC m=+156.420718544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls") pod "image-registry-b796f6cf6-n4trr" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a") : secret "image-registry-tls" not found May 28 17:12:25.152248 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:25.152210 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p" event={"ID":"b0750930-f62e-4151-83ec-ee92469cfa02","Type":"ContainerStarted","Data":"d538c1014d9b7c19986530226e3ada476b6c87b3fc7aa7b903052e9d24aa4470"} May 28 17:12:25.178061 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:25.178004 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-spj4p" podStartSLOduration=1.6117959819999998 podStartE2EDuration="3.177989724s" podCreationTimestamp="2026-05-28 17:12:22 +0000 UTC" firstStartedPulling="2026-05-28 17:12:22.988326522 +0000 UTC m=+152.900719153" lastFinishedPulling="2026-05-28 17:12:24.554520256 +0000 UTC m=+154.466912895" observedRunningTime="2026-05-28 17:12:25.177978971 +0000 UTC m=+155.090371625" watchObservedRunningTime="2026-05-28 17:12:25.177989724 +0000 UTC m=+155.090382378" May 28 17:12:26.156148 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:26.156107 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" event={"ID":"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d","Type":"ContainerStarted","Data":"4016fce2fee01b407be872678d7b778a76b0a8c80735103118f3e57c4fcd8f21"} May 28 17:12:26.157560 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:26.157514 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" event={"ID":"907ac90b-550c-4975-b881-6e7398ee2437","Type":"ContainerStarted","Data":"523de421b6d74614de433fe92e0cd969eac133e060198e4994887f1895f47de9"} May 28 17:12:26.174065 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:26.174014 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" podStartSLOduration=1.816078263 podStartE2EDuration="4.174001326s" podCreationTimestamp="2026-05-28 17:12:22 +0000 UTC" firstStartedPulling="2026-05-28 17:12:23.217282213 +0000 UTC m=+153.129674850" lastFinishedPulling="2026-05-28 17:12:25.575205266 +0000 UTC m=+155.487597913" observedRunningTime="2026-05-28 17:12:26.173528947 +0000 UTC m=+156.085921614" watchObservedRunningTime="2026-05-28 17:12:26.174001326 +0000 UTC m=+156.086393978" May 28 17:12:26.196818 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:26.196758 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" podStartSLOduration=1.83010578 podStartE2EDuration="4.196739793s" podCreationTimestamp="2026-05-28 17:12:22 +0000 UTC" firstStartedPulling="2026-05-28 17:12:23.206638289 +0000 UTC m=+153.119030919" lastFinishedPulling="2026-05-28 17:12:25.573272301 +0000 UTC m=+155.485664932" observedRunningTime="2026-05-28 17:12:26.19585232 +0000 UTC m=+156.108244974" watchObservedRunningTime="2026-05-28 17:12:26.196739793 +0000 UTC m=+156.109132447" May 28 17:12:26.426002 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:26.425891 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:26.426002 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:26.425939 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:26.426189 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:26.426061 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 17:12:26.426189 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:26.426178 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:30.426155717 +0000 UTC m=+160.338548363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : secret "router-metrics-certs-default" not found May 28 17:12:26.426275 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:26.426260 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:30.426240554 +0000 UTC m=+160.338633188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : configmap references non-existent config key: service-ca.crt May 28 17:12:26.526864 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:26.526822 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:26.527038 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:26.526983 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 17:12:26.527038 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:26.527003 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b796f6cf6-n4trr: secret "image-registry-tls" not found May 28 17:12:26.527120 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:26.527071 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls podName:86aa82f8-5676-47b2-8cb2-03731a76fb6a nodeName:}" failed. No retries permitted until 2026-05-28 17:12:30.527055055 +0000 UTC m=+160.439447685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls") pod "image-registry-b796f6cf6-n4trr" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a") : secret "image-registry-tls" not found May 28 17:12:27.463136 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:27.463101 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-rqq5s" podUID="9e5b6585-37d1-4e39-8599-509b085c834a" May 28 17:12:27.473388 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:27.473356 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-h68jj" podUID="b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c" May 28 17:12:27.646993 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:27.646935 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lsl89" podUID="e04539f3-03d0-41ac-8c13-0f67edc8e25c" May 28 17:12:27.833286 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:27.833260 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4frtv_37b8dbdd-d75f-47e5-8776-fbb29a0f2104/dns-node-resolver/0.log" May 28 17:12:28.161814 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:28.161736 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:12:29.020184 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:29.020152 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-szzhz_a73de68c-3175-4736-bf34-52f955c17308/node-ca/0.log" May 28 17:12:30.458419 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:30.458381 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:30.458419 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:30.458424 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:30.458891 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:30.458542 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 17:12:30.458891 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:30.458573 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:38.458559515 +0000 UTC m=+168.370952145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : configmap references non-existent config key: service-ca.crt May 28 17:12:30.458891 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:30.458621 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:38.45860696 +0000 UTC m=+168.370999590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : secret "router-metrics-certs-default" not found May 28 17:12:30.558813 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:30.558778 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:30.558991 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:30.558919 2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 17:12:30.558991 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:30.558935 2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b796f6cf6-n4trr: secret "image-registry-tls" not found May 28 17:12:30.559076 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:30.559003 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls podName:86aa82f8-5676-47b2-8cb2-03731a76fb6a nodeName:}" failed. No retries permitted until 2026-05-28 17:12:38.558986486 +0000 UTC m=+168.471379125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls") pod "image-registry-b796f6cf6-n4trr" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a") : secret "image-registry-tls" not found May 28 17:12:32.373805 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:32.373771 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:12:32.374254 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:32.373850 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:12:32.374254 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:32.373938 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 17:12:32.374254 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:32.373969 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 17:12:32.374254 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:32.374012 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls podName:b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c nodeName:}" failed. No retries permitted until 2026-05-28 17:14:34.373999178 +0000 UTC m=+284.286391809 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls") pod "dns-default-h68jj" (UID: "b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c") : secret "dns-default-metrics-tls" not found May 28 17:12:32.374254 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:32.374048 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert podName:9e5b6585-37d1-4e39-8599-509b085c834a nodeName:}" failed. No retries permitted until 2026-05-28 17:14:34.374026819 +0000 UTC m=+284.286419460 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert") pod "ingress-canary-rqq5s" (UID: "9e5b6585-37d1-4e39-8599-509b085c834a") : secret "canary-serving-cert" not found May 28 17:12:38.520773 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:38.520668 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:38.520773 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:38.520722 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:38.521311 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:12:38.520835 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle podName:ca21983b-3973-4275-86f0-65d6fb0410f1 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:54.520816815 +0000 UTC m=+184.433209445 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle") pod "router-default-7d5f85c4d6-gs4zx" (UID: "ca21983b-3973-4275-86f0-65d6fb0410f1") : configmap references non-existent config key: service-ca.crt May 28 17:12:38.523084 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:38.523060 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca21983b-3973-4275-86f0-65d6fb0410f1-metrics-certs\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:38.622220 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:38.622167 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:38.624514 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:38.624488 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"image-registry-b796f6cf6-n4trr\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:38.637477 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:38.637450 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:12:38.694801 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:38.694763 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:38.819361 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:38.819261 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b796f6cf6-n4trr"] May 28 17:12:38.824659 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:12:38.824621 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86aa82f8_5676_47b2_8cb2_03731a76fb6a.slice/crio-a09d0cf7a17745ed2b59d958ae199bc813c12cf74c263aa787f9ee2352a4d3d9 WatchSource:0}: Error finding container a09d0cf7a17745ed2b59d958ae199bc813c12cf74c263aa787f9ee2352a4d3d9: Status 404 returned error can't find the container with id a09d0cf7a17745ed2b59d958ae199bc813c12cf74c263aa787f9ee2352a4d3d9 May 28 17:12:39.188824 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:39.188747 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" event={"ID":"86aa82f8-5676-47b2-8cb2-03731a76fb6a","Type":"ContainerStarted","Data":"f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86"} May 28 17:12:39.188824 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:39.188787 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" event={"ID":"86aa82f8-5676-47b2-8cb2-03731a76fb6a","Type":"ContainerStarted","Data":"a09d0cf7a17745ed2b59d958ae199bc813c12cf74c263aa787f9ee2352a4d3d9"} May 28 17:12:39.189040 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:39.188900 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:12:39.211382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:39.211328 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" podStartSLOduration=17.211313132 podStartE2EDuration="17.211313132s" podCreationTimestamp="2026-05-28 17:12:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:12:39.210497286 +0000 UTC m=+169.122889945" watchObservedRunningTime="2026-05-28 17:12:39.211313132 +0000 UTC m=+169.123705815" May 28 17:12:41.637184 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:41.637135 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h68jj" May 28 17:12:54.544198 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:54.544164 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:54.544775 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:54.544752 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca21983b-3973-4275-86f0-65d6fb0410f1-service-ca-bundle\") pod \"router-default-7d5f85c4d6-gs4zx\" (UID: \"ca21983b-3973-4275-86f0-65d6fb0410f1\") " pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:54.762657 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:54.762631 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zvwff\"" May 28 17:12:54.770767 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:54.770754 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:54.891404 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:54.891367 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d5f85c4d6-gs4zx"] May 28 17:12:54.894551 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:12:54.894525 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca21983b_3973_4275_86f0_65d6fb0410f1.slice/crio-0d1a8e554b98d3273a8e30786106a1f49ce604a7f0febb835bc1dac2d32de229 WatchSource:0}: Error finding container 0d1a8e554b98d3273a8e30786106a1f49ce604a7f0febb835bc1dac2d32de229: Status 404 returned error can't find the container with id 0d1a8e554b98d3273a8e30786106a1f49ce604a7f0febb835bc1dac2d32de229 May 28 17:12:55.226148 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:55.226065 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" event={"ID":"ca21983b-3973-4275-86f0-65d6fb0410f1","Type":"ContainerStarted","Data":"9d78159a0b765f13f298776bb926add59f1a3535d664b52a724a60e694bd60b2"} May 28 17:12:55.226148 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:55.226103 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" event={"ID":"ca21983b-3973-4275-86f0-65d6fb0410f1","Type":"ContainerStarted","Data":"0d1a8e554b98d3273a8e30786106a1f49ce604a7f0febb835bc1dac2d32de229"} May 28 17:12:55.245194 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:55.245151 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" podStartSLOduration=33.245137355 podStartE2EDuration="33.245137355s" podCreationTimestamp="2026-05-28 17:12:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:12:55.244032907 +0000 UTC m=+185.156425560" watchObservedRunningTime="2026-05-28 17:12:55.245137355 +0000 UTC m=+185.157530008" May 28 17:12:55.771588 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:55.771551 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:55.774042 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:55.774009 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:56.228783 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:56.228708 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:56.229981 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:56.229962 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d5f85c4d6-gs4zx" May 28 17:12:58.699228 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:58.699185 2566 patch_prober.go:28] interesting pod/image-registry-b796f6cf6-n4trr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 28 17:12:58.699606 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:12:58.699248 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" podUID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 17:13:00.195821 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.195789 2566 patch_prober.go:28] interesting pod/image-registry-b796f6cf6-n4trr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 28 17:13:00.196267 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.195836 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" podUID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 17:13:00.941443 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.941405 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qc67w"] May 28 17:13:00.944663 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.944586 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:00.947279 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.947253 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 28 17:13:00.948349 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.948271 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 28 17:13:00.948349 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.948295 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 28 17:13:00.948545 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.948347 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 28 17:13:00.948545 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.948347 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wlzz5\"" May 28 17:13:00.952979 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.952936 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b796f6cf6-n4trr"] May 28 17:13:00.957051 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.957008 2566 patch_prober.go:28] interesting pod/image-registry-b796f6cf6-n4trr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 28 17:13:00.957176 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.957049 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" podUID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 17:13:00.969666 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.969630 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qc67w"] May 28 17:13:00.990717 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.990689 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms9db\" (UniqueName: \"kubernetes.io/projected/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-kube-api-access-ms9db\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:00.990865 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.990725 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-data-volume\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:00.990865 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.990743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-crio-socket\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:00.990865 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.990762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:00.990989 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:00.990874 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.021677 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.021644 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld"] May 28 17:13:01.023397 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.023377 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" May 28 17:13:01.026392 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.026373 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" May 28 17:13:01.026493 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.026402 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-v2jdf\"" May 28 17:13:01.034428 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.034409 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-q6g5h"] May 28 17:13:01.036071 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.036057 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-q6g5h" May 28 17:13:01.038486 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.038464 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wk4bn\"" May 28 17:13:01.038601 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.038563 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 28 17:13:01.039032 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.039016 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 28 17:13:01.039828 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.039808 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-747fdf66db-spdfc"] May 28 17:13:01.041439 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.041421 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.045791 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.045764 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld"] May 28 17:13:01.054079 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.054059 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-q6g5h"] May 28 17:13:01.069631 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.066718 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-747fdf66db-spdfc"] May 28 17:13:01.091294 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091262 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/960a36d4-962b-465b-8376-a1c6e1101a44-image-registry-private-configuration\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.091294 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091296 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/960a36d4-962b-465b-8376-a1c6e1101a44-registry-certificates\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.091498 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091319 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-data-volume\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.091498 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/960a36d4-962b-465b-8376-a1c6e1101a44-installation-pull-secrets\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.091498 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091369 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-crio-socket\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.091498 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091478 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-crio-socket\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.091498 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091479 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.091702 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091519 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/960a36d4-962b-465b-8376-a1c6e1101a44-ca-trust-extracted\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.091702 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-registry-tls\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.091702 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091566 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/960a36d4-962b-465b-8376-a1c6e1101a44-trusted-ca\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.091702 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091581 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbcw9\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-kube-api-access-sbcw9\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.091702 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c4b5ab3a-ef21-4f99-8d9b-6215df555111-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-f69ld\" (UID: \"c4b5ab3a-ef21-4f99-8d9b-6215df555111\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" May 28 17:13:01.091702 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091636 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-data-volume\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.091702 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091685 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.091932 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091737 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmkrp\" (UniqueName: \"kubernetes.io/projected/d1f7c3ef-49d9-44ed-b0ed-036939e5ef44-kube-api-access-bmkrp\") pod \"downloads-57bd96b888-q6g5h\" (UID: \"d1f7c3ef-49d9-44ed-b0ed-036939e5ef44\") " pod="openshift-console/downloads-57bd96b888-q6g5h" May 28 17:13:01.091932 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091754 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-bound-sa-token\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.091932 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091776 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ms9db\" (UniqueName: \"kubernetes.io/projected/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-kube-api-access-ms9db\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.091932 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.091897 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.094278 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.094253 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.103568 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.103548 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms9db\" (UniqueName: \"kubernetes.io/projected/ed52b9e1-36c2-4a65-9f38-edc2790fc1e0-kube-api-access-ms9db\") pod \"insights-runtime-extractor-qc67w\" (UID: \"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0\") " pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.192414 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192332 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-registry-tls\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.192414 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192367 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/960a36d4-962b-465b-8376-a1c6e1101a44-trusted-ca\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.192414 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192390 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sbcw9\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-kube-api-access-sbcw9\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.192692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192418 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c4b5ab3a-ef21-4f99-8d9b-6215df555111-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-f69ld\" (UID: \"c4b5ab3a-ef21-4f99-8d9b-6215df555111\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" May 28 17:13:01.192692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192492 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bmkrp\" (UniqueName: \"kubernetes.io/projected/d1f7c3ef-49d9-44ed-b0ed-036939e5ef44-kube-api-access-bmkrp\") pod \"downloads-57bd96b888-q6g5h\" (UID: \"d1f7c3ef-49d9-44ed-b0ed-036939e5ef44\") " pod="openshift-console/downloads-57bd96b888-q6g5h" May 28 17:13:01.192692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192517 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-bound-sa-token\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.192692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192549 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/960a36d4-962b-465b-8376-a1c6e1101a44-image-registry-private-configuration\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.192692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192572 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/960a36d4-962b-465b-8376-a1c6e1101a44-registry-certificates\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.192692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192599 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/960a36d4-962b-465b-8376-a1c6e1101a44-installation-pull-secrets\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.192692 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.192648 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/960a36d4-962b-465b-8376-a1c6e1101a44-ca-trust-extracted\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.193073 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.193055 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/960a36d4-962b-465b-8376-a1c6e1101a44-ca-trust-extracted\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.193411 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.193378 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/960a36d4-962b-465b-8376-a1c6e1101a44-trusted-ca\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.193890 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.193868 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/960a36d4-962b-465b-8376-a1c6e1101a44-registry-certificates\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.195021 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.194997 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c4b5ab3a-ef21-4f99-8d9b-6215df555111-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-f69ld\" (UID: \"c4b5ab3a-ef21-4f99-8d9b-6215df555111\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" May 28 17:13:01.195149 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.195131 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-registry-tls\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.195534 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.195513 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/960a36d4-962b-465b-8376-a1c6e1101a44-installation-pull-secrets\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.195624 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.195585 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/960a36d4-962b-465b-8376-a1c6e1101a44-image-registry-private-configuration\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.202854 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.202824 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-bound-sa-token\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.203347 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.203327 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmkrp\" (UniqueName: \"kubernetes.io/projected/d1f7c3ef-49d9-44ed-b0ed-036939e5ef44-kube-api-access-bmkrp\") pod \"downloads-57bd96b888-q6g5h\" (UID: \"d1f7c3ef-49d9-44ed-b0ed-036939e5ef44\") " pod="openshift-console/downloads-57bd96b888-q6g5h" May 28 17:13:01.203896 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.203879 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbcw9\" (UniqueName: \"kubernetes.io/projected/960a36d4-962b-465b-8376-a1c6e1101a44-kube-api-access-sbcw9\") pod \"image-registry-747fdf66db-spdfc\" (UID: \"960a36d4-962b-465b-8376-a1c6e1101a44\") " pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.255285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.255255 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qc67w" May 28 17:13:01.332322 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.332264 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" May 28 17:13:01.344052 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.344030 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-q6g5h" May 28 17:13:01.350801 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.350775 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:01.383819 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.383767 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qc67w"] May 28 17:13:01.387271 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:01.387231 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded52b9e1_36c2_4a65_9f38_edc2790fc1e0.slice/crio-69ed4bef86f6bcb00818ad5f63a7c7d728698d9ca3c8209a16b585060b4225e0 WatchSource:0}: Error finding container 69ed4bef86f6bcb00818ad5f63a7c7d728698d9ca3c8209a16b585060b4225e0: Status 404 returned error can't find the container with id 69ed4bef86f6bcb00818ad5f63a7c7d728698d9ca3c8209a16b585060b4225e0 May 28 17:13:01.484151 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:01.484092 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b5ab3a_ef21_4f99_8d9b_6215df555111.slice/crio-0b51591fe7975f5c19d9459f345c9e775292e58e46f36aa9656f1a13cccb72cf WatchSource:0}: Error finding container 0b51591fe7975f5c19d9459f345c9e775292e58e46f36aa9656f1a13cccb72cf: Status 404 returned error can't find the container with id 0b51591fe7975f5c19d9459f345c9e775292e58e46f36aa9656f1a13cccb72cf May 28 17:13:01.484658 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.484608 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld"] May 28 17:13:01.503738 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.503711 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-q6g5h"] May 28 17:13:01.508136 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:01.508106 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f7c3ef_49d9_44ed_b0ed_036939e5ef44.slice/crio-66e4478dbd123914fed130931cb26876650ecb15ba1e83222c54ebf69aa66dce WatchSource:0}: Error finding container 66e4478dbd123914fed130931cb26876650ecb15ba1e83222c54ebf69aa66dce: Status 404 returned error can't find the container with id 66e4478dbd123914fed130931cb26876650ecb15ba1e83222c54ebf69aa66dce May 28 17:13:01.525991 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:01.525966 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-747fdf66db-spdfc"] May 28 17:13:01.528318 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:01.528294 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960a36d4_962b_465b_8376_a1c6e1101a44.slice/crio-e8c70e1dc6d2cb9a96b19f2e5d531690fe55b4c895e9e43ebf3a38f54118916b WatchSource:0}: Error finding container e8c70e1dc6d2cb9a96b19f2e5d531690fe55b4c895e9e43ebf3a38f54118916b: Status 404 returned error can't find the container with id e8c70e1dc6d2cb9a96b19f2e5d531690fe55b4c895e9e43ebf3a38f54118916b May 28 17:13:02.246453 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:02.246418 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-747fdf66db-spdfc" event={"ID":"960a36d4-962b-465b-8376-a1c6e1101a44","Type":"ContainerStarted","Data":"1a2d375c092cafe0409088bc32d57b702329d10f397e9b6d79208ebf4a7040eb"} May 28 17:13:02.246842 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:02.246461 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-747fdf66db-spdfc" event={"ID":"960a36d4-962b-465b-8376-a1c6e1101a44","Type":"ContainerStarted","Data":"e8c70e1dc6d2cb9a96b19f2e5d531690fe55b4c895e9e43ebf3a38f54118916b"} May 28 17:13:02.246842 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:02.246515 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:02.248141 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:02.248105 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-q6g5h" event={"ID":"d1f7c3ef-49d9-44ed-b0ed-036939e5ef44","Type":"ContainerStarted","Data":"66e4478dbd123914fed130931cb26876650ecb15ba1e83222c54ebf69aa66dce"} May 28 17:13:02.249966 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:02.249927 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qc67w" event={"ID":"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0","Type":"ContainerStarted","Data":"46271f834446a44dcb7bc6b6b83957840118b2024d4d7f26d7c11d82ac001162"} May 28 17:13:02.250113 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:02.250085 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qc67w" event={"ID":"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0","Type":"ContainerStarted","Data":"69ed4bef86f6bcb00818ad5f63a7c7d728698d9ca3c8209a16b585060b4225e0"} May 28 17:13:02.251452 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:02.251427 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" event={"ID":"c4b5ab3a-ef21-4f99-8d9b-6215df555111","Type":"ContainerStarted","Data":"0b51591fe7975f5c19d9459f345c9e775292e58e46f36aa9656f1a13cccb72cf"} May 28 17:13:02.276027 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:02.275984 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-747fdf66db-spdfc" podStartSLOduration=1.275967365 podStartE2EDuration="1.275967365s" podCreationTimestamp="2026-05-28 17:13:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:13:02.274803623 +0000 UTC m=+192.187196287" watchObservedRunningTime="2026-05-28 17:13:02.275967365 +0000 UTC m=+192.188360017" May 28 17:13:03.257916 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:03.257879 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qc67w" event={"ID":"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0","Type":"ContainerStarted","Data":"69df63b2584acedefcce8e34add1185e6e0c06746ed03864d8bd3aebe2ebf8e4"} May 28 17:13:03.259435 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:03.259384 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" event={"ID":"c4b5ab3a-ef21-4f99-8d9b-6215df555111","Type":"ContainerStarted","Data":"395ff671c501ccdcbfc4d1dbe27ef7725d9f2a1f110fea8eadf60c816c349237"} May 28 17:13:03.259914 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:03.259829 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" May 28 17:13:03.266431 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:03.266409 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" May 28 17:13:03.275664 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:03.275615 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-f69ld" podStartSLOduration=1.073744888 podStartE2EDuration="2.275599108s" podCreationTimestamp="2026-05-28 17:13:01 +0000 UTC" firstStartedPulling="2026-05-28 17:13:01.486261931 +0000 UTC m=+191.398654573" lastFinishedPulling="2026-05-28 17:13:02.688116162 +0000 UTC m=+192.600508793" observedRunningTime="2026-05-28 17:13:03.274838788 +0000 UTC m=+193.187231441" watchObservedRunningTime="2026-05-28 17:13:03.275599108 +0000 UTC m=+193.187991763" May 28 17:13:04.264109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:04.264069 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qc67w" event={"ID":"ed52b9e1-36c2-4a65-9f38-edc2790fc1e0","Type":"ContainerStarted","Data":"ae07c366fac793c1271570296265a8a3e5b4c86cc88340dfcc53ce095c05e402"} May 28 17:13:04.284066 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:04.284005 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qc67w" podStartSLOduration=1.8031912719999998 podStartE2EDuration="4.283987069s" podCreationTimestamp="2026-05-28 17:13:00 +0000 UTC" firstStartedPulling="2026-05-28 17:13:01.484213017 +0000 UTC m=+191.396605655" lastFinishedPulling="2026-05-28 17:13:03.965008808 +0000 UTC m=+193.877401452" observedRunningTime="2026-05-28 17:13:04.282580903 +0000 UTC m=+194.194973557" watchObservedRunningTime="2026-05-28 17:13:04.283987069 +0000 UTC m=+194.196379721" May 28 17:13:10.958549 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:10.958506 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:13:13.179483 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.179445 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zwtlz"] May 28 17:13:13.184410 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.184385 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.187001 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.186974 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 28 17:13:13.187132 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.186973 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 28 17:13:13.187299 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.187280 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 28 17:13:13.187461 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.187444 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 28 17:13:13.187922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.187902 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8795h\"" May 28 17:13:13.188051 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.187988 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 28 17:13:13.189189 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.189165 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 28 17:13:13.307004 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.306970 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.307004 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.307004 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-sys\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.307226 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.307101 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-textfile\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.307226 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.307137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-root\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.307226 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.307175 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-tls\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.307226 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.307219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37d65077-20e2-4f30-9f1f-42a66f11c959-metrics-client-ca\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.307395 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.307240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-accelerators-collector-config\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.307395 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.307273 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-wtmp\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.307395 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.307291 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58fq5\" (UniqueName: \"kubernetes.io/projected/37d65077-20e2-4f30-9f1f-42a66f11c959-kube-api-access-58fq5\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408452 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408411 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408628 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408462 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-sys\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408628 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408515 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-sys\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408628 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408580 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-textfile\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408628 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408603 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-root\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408823 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408630 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-tls\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408823 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408655 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37d65077-20e2-4f30-9f1f-42a66f11c959-metrics-client-ca\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408823 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408684 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-accelerators-collector-config\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408823 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408714 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-wtmp\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408823 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408718 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-root\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408823 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408730 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-58fq5\" (UniqueName: \"kubernetes.io/projected/37d65077-20e2-4f30-9f1f-42a66f11c959-kube-api-access-58fq5\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.408823 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:13:13.408768 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found May 28 17:13:13.409199 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:13:13.408848 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-tls podName:37d65077-20e2-4f30-9f1f-42a66f11c959 nodeName:}" failed. No retries permitted until 2026-05-28 17:13:13.90882491 +0000 UTC m=+203.821217547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-tls") pod "node-exporter-zwtlz" (UID: "37d65077-20e2-4f30-9f1f-42a66f11c959") : secret "node-exporter-tls" not found May 28 17:13:13.409199 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.408940 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-textfile\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.409199 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.409097 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-wtmp\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.409403 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.409368 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-accelerators-collector-config\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.409525 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.409402 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37d65077-20e2-4f30-9f1f-42a66f11c959-metrics-client-ca\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.411353 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.411330 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.419135 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.419114 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58fq5\" (UniqueName: \"kubernetes.io/projected/37d65077-20e2-4f30-9f1f-42a66f11c959-kube-api-access-58fq5\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.914152 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.914114 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-tls\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:13.916660 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:13.916637 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/37d65077-20e2-4f30-9f1f-42a66f11c959-node-exporter-tls\") pod \"node-exporter-zwtlz\" (UID: \"37d65077-20e2-4f30-9f1f-42a66f11c959\") " pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:14.096397 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.096353 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zwtlz" May 28 17:13:14.321067 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.321037 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 17:13:14.326051 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.326025 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.335870 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.335852 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" May 28 17:13:14.336269 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.336249 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" May 28 17:13:14.336389 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.336254 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" May 28 17:13:14.336660 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.336640 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" May 28 17:13:14.336660 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.336653 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" May 28 17:13:14.336799 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.336642 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" May 28 17:13:14.336856 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.336808 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" May 28 17:13:14.336856 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.336839 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tt9wm\"" May 28 17:13:14.336969 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.336940 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" May 28 17:13:14.337087 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.337067 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" May 28 17:13:14.361312 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.361287 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 17:13:14.419298 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419264 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f64ab61a-2865-46ac-822e-8a5d31318958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419457 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419310 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-config-volume\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419457 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419457 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419364 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419457 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f64ab61a-2865-46ac-822e-8a5d31318958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419457 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f64ab61a-2865-46ac-822e-8a5d31318958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419467 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfdc6\" (UniqueName: \"kubernetes.io/projected/f64ab61a-2865-46ac-822e-8a5d31318958-kube-api-access-xfdc6\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-web-config\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419620 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f64ab61a-2865-46ac-822e-8a5d31318958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.419713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f64ab61a-2865-46ac-822e-8a5d31318958-config-out\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.420034 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.419710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521065 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521025 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f64ab61a-2865-46ac-822e-8a5d31318958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521255 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521074 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-config-volume\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521255 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521213 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521380 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521262 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521380 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521290 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f64ab61a-2865-46ac-822e-8a5d31318958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521380 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521316 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f64ab61a-2865-46ac-822e-8a5d31318958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521380 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521356 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521579 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521382 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xfdc6\" (UniqueName: \"kubernetes.io/projected/f64ab61a-2865-46ac-822e-8a5d31318958-kube-api-access-xfdc6\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521771 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521720 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f64ab61a-2865-46ac-822e-8a5d31318958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.521889 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521847 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-web-config\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.522297 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521903 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f64ab61a-2865-46ac-822e-8a5d31318958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.522297 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.521932 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.522297 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.522085 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f64ab61a-2865-46ac-822e-8a5d31318958-config-out\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.522297 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.522144 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.522584 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.522316 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f64ab61a-2865-46ac-822e-8a5d31318958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.522647 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.522603 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f64ab61a-2865-46ac-822e-8a5d31318958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.524558 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.524533 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.524743 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.524714 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.524847 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.524757 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-config-volume\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.525412 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.525376 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.525615 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.525590 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f64ab61a-2865-46ac-822e-8a5d31318958-config-out\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.525803 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.525781 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f64ab61a-2865-46ac-822e-8a5d31318958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.525981 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.525960 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-web-config\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.526588 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.526569 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.527079 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.527058 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f64ab61a-2865-46ac-822e-8a5d31318958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.530066 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.530048 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfdc6\" (UniqueName: \"kubernetes.io/projected/f64ab61a-2865-46ac-822e-8a5d31318958-kube-api-access-xfdc6\") pod \"alertmanager-main-0\" (UID: \"f64ab61a-2865-46ac-822e-8a5d31318958\") " pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:14.636389 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:14.636308 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 28 17:13:15.239954 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.239901 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b9cd59bcd-z848k"] May 28 17:13:15.244957 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.244918 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.247659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.247631 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" May 28 17:13:15.247790 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.247663 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" May 28 17:13:15.247853 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.247790 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8bvnr5lnpqrvk\"" May 28 17:13:15.247853 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.247801 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-k84dn\"" May 28 17:13:15.248081 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.248061 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" May 28 17:13:15.248179 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.248081 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" May 28 17:13:15.248179 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.248134 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" May 28 17:13:15.255888 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.255869 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b9cd59bcd-z848k"] May 28 17:13:15.331121 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.331082 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2682f774-cded-415f-a4a3-bafb74e0b6b3-metrics-client-ca\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.331555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.331161 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.331555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.331193 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-tls\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.331555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.331249 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.331555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.331317 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.331555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.331428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-grpc-tls\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.331555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.331467 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75qdq\" (UniqueName: \"kubernetes.io/projected/2682f774-cded-415f-a4a3-bafb74e0b6b3-kube-api-access-75qdq\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.331555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.331525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.432674 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.432637 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.432674 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.432677 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-tls\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.432922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.432697 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.432922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.432747 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.432922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.432807 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-grpc-tls\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.432922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.432833 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-75qdq\" (UniqueName: \"kubernetes.io/projected/2682f774-cded-415f-a4a3-bafb74e0b6b3-kube-api-access-75qdq\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.432922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.432879 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.433193 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.432965 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2682f774-cded-415f-a4a3-bafb74e0b6b3-metrics-client-ca\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.433778 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.433697 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2682f774-cded-415f-a4a3-bafb74e0b6b3-metrics-client-ca\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.435804 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.435777 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.435917 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.435804 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-tls\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.435917 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.435861 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.436159 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.436132 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.436257 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.436219 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.436600 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.436581 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2682f774-cded-415f-a4a3-bafb74e0b6b3-secret-grpc-tls\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.440955 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.440917 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75qdq\" (UniqueName: \"kubernetes.io/projected/2682f774-cded-415f-a4a3-bafb74e0b6b3-kube-api-access-75qdq\") pod \"thanos-querier-5b9cd59bcd-z848k\" (UID: \"2682f774-cded-415f-a4a3-bafb74e0b6b3\") " pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:15.556097 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:15.556010 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:16.747850 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:16.747813 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d65077_20e2_4f30_9f1f_42a66f11c959.slice/crio-cef5cc6eb6406a91617d62a0ed8bed4c88c608b9f21f38c9827ec5508a1108ec WatchSource:0}: Error finding container cef5cc6eb6406a91617d62a0ed8bed4c88c608b9f21f38c9827ec5508a1108ec: Status 404 returned error can't find the container with id cef5cc6eb6406a91617d62a0ed8bed4c88c608b9f21f38c9827ec5508a1108ec May 28 17:13:16.882177 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:16.882149 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b9cd59bcd-z848k"] May 28 17:13:16.883195 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:16.883037 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2682f774_cded_415f_a4a3_bafb74e0b6b3.slice/crio-b183fde9d1d865720e04e262b1f23e5d54531a582c90ee0e5278f65e02db4ea0 WatchSource:0}: Error finding container b183fde9d1d865720e04e262b1f23e5d54531a582c90ee0e5278f65e02db4ea0: Status 404 returned error can't find the container with id b183fde9d1d865720e04e262b1f23e5d54531a582c90ee0e5278f65e02db4ea0 May 28 17:13:16.898540 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:16.898509 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 17:13:16.901345 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:16.901318 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64ab61a_2865_46ac_822e_8a5d31318958.slice/crio-6d542571a6ce48bd90d9461cd6bb7acef24da981f62fc0a7645eeb8b65fd2223 WatchSource:0}: Error finding container 6d542571a6ce48bd90d9461cd6bb7acef24da981f62fc0a7645eeb8b65fd2223: Status 404 returned error can't find the container with id 6d542571a6ce48bd90d9461cd6bb7acef24da981f62fc0a7645eeb8b65fd2223 May 28 17:13:17.302059 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.302016 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-q6g5h" event={"ID":"d1f7c3ef-49d9-44ed-b0ed-036939e5ef44","Type":"ContainerStarted","Data":"8e06c79b7b76d2222bffd537c72510729857e0039d86a801bc85660754414d64"} May 28 17:13:17.302686 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.302645 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-q6g5h" May 28 17:13:17.304389 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.304364 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f64ab61a-2865-46ac-822e-8a5d31318958","Type":"ContainerStarted","Data":"6d542571a6ce48bd90d9461cd6bb7acef24da981f62fc0a7645eeb8b65fd2223"} May 28 17:13:17.306216 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.306182 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" event={"ID":"2682f774-cded-415f-a4a3-bafb74e0b6b3","Type":"ContainerStarted","Data":"b183fde9d1d865720e04e262b1f23e5d54531a582c90ee0e5278f65e02db4ea0"} May 28 17:13:17.307980 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.307915 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zwtlz" event={"ID":"37d65077-20e2-4f30-9f1f-42a66f11c959","Type":"ContainerStarted","Data":"cef5cc6eb6406a91617d62a0ed8bed4c88c608b9f21f38c9827ec5508a1108ec"} May 28 17:13:17.308538 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.308507 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-q6g5h" May 28 17:13:17.321290 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.321243 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-q6g5h" podStartSLOduration=0.991706961 podStartE2EDuration="16.321223904s" podCreationTimestamp="2026-05-28 17:13:01 +0000 UTC" firstStartedPulling="2026-05-28 17:13:01.509827764 +0000 UTC m=+191.422220395" lastFinishedPulling="2026-05-28 17:13:16.839344697 +0000 UTC m=+206.751737338" observedRunningTime="2026-05-28 17:13:17.319253128 +0000 UTC m=+207.231645781" watchObservedRunningTime="2026-05-28 17:13:17.321223904 +0000 UTC m=+207.233616559" May 28 17:13:17.606003 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.604414 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56569f4c5-htn9m"] May 28 17:13:17.612217 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.609261 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.614066 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.613008 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" May 28 17:13:17.614066 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.613289 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 28 17:13:17.615021 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.614466 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" May 28 17:13:17.615021 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.614569 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xbqpq\"" May 28 17:13:17.615021 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.614735 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4dvbv030dh58b\"" May 28 17:13:17.615021 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.614779 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" May 28 17:13:17.622788 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.621841 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56569f4c5-htn9m"] May 28 17:13:17.756891 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.756846 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-secret-metrics-server-client-certs\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.757238 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.756976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.757238 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.757031 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-audit-log\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.757238 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.757096 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwc57\" (UniqueName: \"kubernetes.io/projected/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-kube-api-access-nwc57\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.757238 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.757137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-secret-metrics-server-tls\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.757238 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.757164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-client-ca-bundle\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.757238 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.757193 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-metrics-server-audit-profiles\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.858727 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.858643 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-secret-metrics-server-tls\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.858727 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.858690 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-client-ca-bundle\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.858727 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.858719 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-metrics-server-audit-profiles\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.858982 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.858779 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-secret-metrics-server-client-certs\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.858982 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.858869 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.858982 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.858915 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-audit-log\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.859112 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.858998 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nwc57\" (UniqueName: \"kubernetes.io/projected/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-kube-api-access-nwc57\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.860297 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.860071 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.860297 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.860250 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-metrics-server-audit-profiles\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.860826 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.860807 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-audit-log\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.862457 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.862409 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-client-ca-bundle\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.863506 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.863466 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-secret-metrics-server-client-certs\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.863607 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.863526 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-secret-metrics-server-tls\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.876180 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.876122 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwc57\" (UniqueName: \"kubernetes.io/projected/e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b-kube-api-access-nwc57\") pod \"metrics-server-56569f4c5-htn9m\" (UID: \"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b\") " pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.926937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.926903 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:17.951727 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.950814 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-std88"] May 28 17:13:17.955642 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.954840 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" May 28 17:13:17.959490 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.959464 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-42hd8\"" May 28 17:13:17.963512 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.959723 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" May 28 17:13:17.963512 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:17.962854 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-std88"] May 28 17:13:18.065272 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:18.065219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70f58936-9848-4a04-b2a8-42e4333c7fec-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-std88\" (UID: \"70f58936-9848-4a04-b2a8-42e4333c7fec\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" May 28 17:13:18.167390 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:18.166799 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70f58936-9848-4a04-b2a8-42e4333c7fec-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-std88\" (UID: \"70f58936-9848-4a04-b2a8-42e4333c7fec\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" May 28 17:13:18.170474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:18.170409 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70f58936-9848-4a04-b2a8-42e4333c7fec-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-std88\" (UID: \"70f58936-9848-4a04-b2a8-42e4333c7fec\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" May 28 17:13:18.283397 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:18.283352 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" May 28 17:13:18.486430 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:18.486377 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56569f4c5-htn9m"] May 28 17:13:18.494350 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:18.494306 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e3d7a0_2fb0_4575_9d09_4a6642dafa4b.slice/crio-0479f9bd41258107333f7559b22e47be0da2d44ea3218a86a149f8189a20082f WatchSource:0}: Error finding container 0479f9bd41258107333f7559b22e47be0da2d44ea3218a86a149f8189a20082f: Status 404 returned error can't find the container with id 0479f9bd41258107333f7559b22e47be0da2d44ea3218a86a149f8189a20082f May 28 17:13:18.502564 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:18.502533 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-std88"] May 28 17:13:18.506574 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:18.506541 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f58936_9848_4a04_b2a8_42e4333c7fec.slice/crio-817a022d48faf6995ab14f6299f3887895ca28dc695a13621db1977b1a8c3e72 WatchSource:0}: Error finding container 817a022d48faf6995ab14f6299f3887895ca28dc695a13621db1977b1a8c3e72: Status 404 returned error can't find the container with id 817a022d48faf6995ab14f6299f3887895ca28dc695a13621db1977b1a8c3e72 May 28 17:13:19.316490 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.316451 2566 generic.go:358] "Generic (PLEG): container finished" podID="37d65077-20e2-4f30-9f1f-42a66f11c959" containerID="9a069ecb8bdb2db8ad38fc7412be193a7fe6c420f971cb4da987ef0f264a5645" exitCode=0 May 28 17:13:19.316970 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.316547 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zwtlz" event={"ID":"37d65077-20e2-4f30-9f1f-42a66f11c959","Type":"ContainerDied","Data":"9a069ecb8bdb2db8ad38fc7412be193a7fe6c420f971cb4da987ef0f264a5645"} May 28 17:13:19.317852 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.317825 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" event={"ID":"70f58936-9848-4a04-b2a8-42e4333c7fec","Type":"ContainerStarted","Data":"817a022d48faf6995ab14f6299f3887895ca28dc695a13621db1977b1a8c3e72"} May 28 17:13:19.319448 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.319424 2566 generic.go:358] "Generic (PLEG): container finished" podID="f64ab61a-2865-46ac-822e-8a5d31318958" containerID="c376fe640a178053cc6586943c17d67afa399ea2f8aabda571e15581e74c0df1" exitCode=0 May 28 17:13:19.319551 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.319518 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f64ab61a-2865-46ac-822e-8a5d31318958","Type":"ContainerDied","Data":"c376fe640a178053cc6586943c17d67afa399ea2f8aabda571e15581e74c0df1"} May 28 17:13:19.320981 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.320934 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" event={"ID":"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b","Type":"ContainerStarted","Data":"0479f9bd41258107333f7559b22e47be0da2d44ea3218a86a149f8189a20082f"} May 28 17:13:19.363117 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.363083 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 17:13:19.392969 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.390777 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.392969 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.390580 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 17:13:19.394062 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.393857 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gjj66\"" May 28 17:13:19.394062 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.393926 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 28 17:13:19.394632 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.394350 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 28 17:13:19.394632 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.394364 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 28 17:13:19.394632 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.394423 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 28 17:13:19.394632 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.394356 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ao54atcd9e2qu\"" May 28 17:13:19.394632 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.394510 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 28 17:13:19.394632 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.394581 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 28 17:13:19.395024 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.394753 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 28 17:13:19.395024 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.394810 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 28 17:13:19.395982 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.395398 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 28 17:13:19.395982 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.395440 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 28 17:13:19.395982 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.395529 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 28 17:13:19.397713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.397692 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 28 17:13:19.481580 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.481546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.481713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.481589 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/32dc81a7-55bb-4586-84f7-c895928c6f26-config-out\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.481713 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.481636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.481815 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.481777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.481875 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.481816 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.481971 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.481924 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-config\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482088 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482035 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482088 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482423 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482423 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482423 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482316 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482423 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482377 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-web-config\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482671 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/32dc81a7-55bb-4586-84f7-c895928c6f26-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482671 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482671 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482559 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482671 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482583 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv64q\" (UniqueName: \"kubernetes.io/projected/32dc81a7-55bb-4586-84f7-c895928c6f26-kube-api-access-qv64q\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482671 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.482899 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.482851 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584104 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584020 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584104 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584068 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584104 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584100 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584132 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-web-config\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584160 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/32dc81a7-55bb-4586-84f7-c895928c6f26-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584199 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584226 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584252 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qv64q\" (UniqueName: \"kubernetes.io/projected/32dc81a7-55bb-4586-84f7-c895928c6f26-kube-api-access-qv64q\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584298 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584360 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584726 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584408 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584726 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584436 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/32dc81a7-55bb-4586-84f7-c895928c6f26-config-out\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584726 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584461 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584726 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584490 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584726 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584517 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584726 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584579 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-config\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584726 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584607 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.584726 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.584638 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.587206 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.585381 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.587206 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.586174 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.587838 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.587490 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.588006 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.587976 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.589482 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.588310 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.589482 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.588718 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-web-config\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.589482 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.588770 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.589482 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.589358 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.589740 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.589630 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/32dc81a7-55bb-4586-84f7-c895928c6f26-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.589740 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.589707 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/32dc81a7-55bb-4586-84f7-c895928c6f26-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.590323 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.590251 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.590416 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.590375 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.591283 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.591242 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-config\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.591659 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.591640 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.591758 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.591742 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.592075 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.592053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/32dc81a7-55bb-4586-84f7-c895928c6f26-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.592357 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.592340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/32dc81a7-55bb-4586-84f7-c895928c6f26-config-out\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.594856 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.594835 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv64q\" (UniqueName: \"kubernetes.io/projected/32dc81a7-55bb-4586-84f7-c895928c6f26-kube-api-access-qv64q\") pod \"prometheus-k8s-0\" (UID: \"32dc81a7-55bb-4586-84f7-c895928c6f26\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.704958 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.703525 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:19.963456 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:19.961137 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 17:13:19.972591 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:13:19.972559 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32dc81a7_55bb_4586_84f7_c895928c6f26.slice/crio-a2b29d86b8ece66183b1647b4c89c25858ecfd5f1365e4e36a9aaa1179708900 WatchSource:0}: Error finding container a2b29d86b8ece66183b1647b4c89c25858ecfd5f1365e4e36a9aaa1179708900: Status 404 returned error can't find the container with id a2b29d86b8ece66183b1647b4c89c25858ecfd5f1365e4e36a9aaa1179708900 May 28 17:13:20.329555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.328035 2566 generic.go:358] "Generic (PLEG): container finished" podID="32dc81a7-55bb-4586-84f7-c895928c6f26" containerID="2a3751f9523985b33587e44647fe91bf8fec323ec343fd38c2b99ed43287e3e7" exitCode=0 May 28 17:13:20.329555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.328162 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"32dc81a7-55bb-4586-84f7-c895928c6f26","Type":"ContainerDied","Data":"2a3751f9523985b33587e44647fe91bf8fec323ec343fd38c2b99ed43287e3e7"} May 28 17:13:20.329555 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.328192 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"32dc81a7-55bb-4586-84f7-c895928c6f26","Type":"ContainerStarted","Data":"a2b29d86b8ece66183b1647b4c89c25858ecfd5f1365e4e36a9aaa1179708900"} May 28 17:13:20.336128 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.336088 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" event={"ID":"2682f774-cded-415f-a4a3-bafb74e0b6b3","Type":"ContainerStarted","Data":"b1e67dcbaf33dd36352b26a88784a0f7f4e4befa3da49fb9d6d5bb0cf52b1514"} May 28 17:13:20.336266 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.336133 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" event={"ID":"2682f774-cded-415f-a4a3-bafb74e0b6b3","Type":"ContainerStarted","Data":"c20a3bec6b19301ddda266356e7c73ba868c5df716e09db13dc14370369878f2"} May 28 17:13:20.336266 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.336150 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" event={"ID":"2682f774-cded-415f-a4a3-bafb74e0b6b3","Type":"ContainerStarted","Data":"bcd6ac6cec5b9455d509620c6f4e0f800fa1e96ec79b2bbf7285aa4b9c7843a1"} May 28 17:13:20.340684 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.339996 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zwtlz" event={"ID":"37d65077-20e2-4f30-9f1f-42a66f11c959","Type":"ContainerStarted","Data":"72ff2e0b4dc49c7760d9c05ba1b30f218421f3c963ede81eddc4299d96b86d29"} May 28 17:13:20.340684 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.340029 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zwtlz" event={"ID":"37d65077-20e2-4f30-9f1f-42a66f11c959","Type":"ContainerStarted","Data":"4ae82b2f889bc50a1292399adffaa4dd2f684e5ce96e7613ff44a64a18906a39"} May 28 17:13:20.370577 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:20.370229 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zwtlz" podStartSLOduration=5.817166922 podStartE2EDuration="7.370213108s" podCreationTimestamp="2026-05-28 17:13:13 +0000 UTC" firstStartedPulling="2026-05-28 17:13:16.749804232 +0000 UTC m=+206.662196870" lastFinishedPulling="2026-05-28 17:13:18.302850418 +0000 UTC m=+208.215243056" observedRunningTime="2026-05-28 17:13:20.369683994 +0000 UTC m=+210.282076645" watchObservedRunningTime="2026-05-28 17:13:20.370213108 +0000 UTC m=+210.282605761" May 28 17:13:23.265073 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:23.265041 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-747fdf66db-spdfc" May 28 17:13:24.357019 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:24.356977 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f64ab61a-2865-46ac-822e-8a5d31318958","Type":"ContainerStarted","Data":"192820f69692ce4fc1a65b7ea79cd708b0ea325083d652c3f53574370e1322f3"} May 28 17:13:24.359459 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:24.358996 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" event={"ID":"e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b","Type":"ContainerStarted","Data":"a258785ee625f8917c6ed3b098b2945c146146dac32471febace02c558b1caf2"} May 28 17:13:24.361642 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:24.361198 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" event={"ID":"70f58936-9848-4a04-b2a8-42e4333c7fec","Type":"ContainerStarted","Data":"d4a4e8bf37fedea157e3e5ef4dcca50bbe35dbdf15f3789a167e275afcdd6bdf"} May 28 17:13:24.361642 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:24.361490 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" May 28 17:13:24.363840 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:24.363815 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" event={"ID":"2682f774-cded-415f-a4a3-bafb74e0b6b3","Type":"ContainerStarted","Data":"05e2239ef9e416d2eaeb79ebcda0b07d4e7ef024d65e7bbd1db611eb7dac9c09"} May 28 17:13:24.368099 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:24.368074 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" May 28 17:13:24.378995 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:24.378928 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" podStartSLOduration=2.396561738 podStartE2EDuration="7.378912039s" podCreationTimestamp="2026-05-28 17:13:17 +0000 UTC" firstStartedPulling="2026-05-28 17:13:18.498357419 +0000 UTC m=+208.410750054" lastFinishedPulling="2026-05-28 17:13:23.480707717 +0000 UTC m=+213.393100355" observedRunningTime="2026-05-28 17:13:24.377665249 +0000 UTC m=+214.290057906" watchObservedRunningTime="2026-05-28 17:13:24.378912039 +0000 UTC m=+214.291304693" May 28 17:13:25.369166 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:25.369099 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f64ab61a-2865-46ac-822e-8a5d31318958","Type":"ContainerStarted","Data":"c139129ef689c52f522dad4686d6913b3b8e033646f4a8f821ca4f0cf06148d5"} May 28 17:13:25.369166 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:25.369148 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f64ab61a-2865-46ac-822e-8a5d31318958","Type":"ContainerStarted","Data":"c95370157984dc794b6385149b23d28d5ff99eb99670e887581ff3cc2f287d52"} May 28 17:13:25.372910 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:25.372699 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" event={"ID":"2682f774-cded-415f-a4a3-bafb74e0b6b3","Type":"ContainerStarted","Data":"1c20aeb2fc6100c88ae9bd071f872d5a418ec97cdb2775a19a5f94e5cf8ac57d"} May 28 17:13:25.974529 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:25.974498 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" podUID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" containerName="registry" containerID="cri-o://f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86" gracePeriod=30 May 28 17:13:26.220337 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.220311 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:13:26.242404 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.242353 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-std88" podStartSLOduration=4.27240071 podStartE2EDuration="9.242338107s" podCreationTimestamp="2026-05-28 17:13:17 +0000 UTC" firstStartedPulling="2026-05-28 17:13:18.510149644 +0000 UTC m=+208.422542278" lastFinishedPulling="2026-05-28 17:13:23.480087029 +0000 UTC m=+213.392479675" observedRunningTime="2026-05-28 17:13:24.397293846 +0000 UTC m=+214.309686499" watchObservedRunningTime="2026-05-28 17:13:26.242338107 +0000 UTC m=+216.154730820" May 28 17:13:26.359040 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359010 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86aa82f8-5676-47b2-8cb2-03731a76fb6a-ca-trust-extracted\") pod \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " May 28 17:13:26.359210 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359066 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") pod \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " May 28 17:13:26.359210 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359085 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-certificates\") pod \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " May 28 17:13:26.359210 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359116 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-bound-sa-token\") pod \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " May 28 17:13:26.359210 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359137 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-installation-pull-secrets\") pod \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " May 28 17:13:26.359210 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359175 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-image-registry-private-configuration\") pod \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " May 28 17:13:26.359210 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359202 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4nbs\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-kube-api-access-w4nbs\") pod \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " May 28 17:13:26.359517 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359228 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-trusted-ca\") pod \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\" (UID: \"86aa82f8-5676-47b2-8cb2-03731a76fb6a\") " May 28 17:13:26.359517 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359487 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "86aa82f8-5676-47b2-8cb2-03731a76fb6a" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 17:13:26.359900 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.359868 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "86aa82f8-5676-47b2-8cb2-03731a76fb6a" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 17:13:26.361550 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.361517 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "86aa82f8-5676-47b2-8cb2-03731a76fb6a" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 17:13:26.361735 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.361585 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-kube-api-access-w4nbs" (OuterVolumeSpecName: "kube-api-access-w4nbs") pod "86aa82f8-5676-47b2-8cb2-03731a76fb6a" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a"). InnerVolumeSpecName "kube-api-access-w4nbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 17:13:26.361735 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.361715 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "86aa82f8-5676-47b2-8cb2-03731a76fb6a" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 17:13:26.361847 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.361799 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "86aa82f8-5676-47b2-8cb2-03731a76fb6a" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 17:13:26.361847 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.361805 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "86aa82f8-5676-47b2-8cb2-03731a76fb6a" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 17:13:26.367756 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.367734 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86aa82f8-5676-47b2-8cb2-03731a76fb6a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "86aa82f8-5676-47b2-8cb2-03731a76fb6a" (UID: "86aa82f8-5676-47b2-8cb2-03731a76fb6a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 17:13:26.378647 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.378618 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f64ab61a-2865-46ac-822e-8a5d31318958","Type":"ContainerStarted","Data":"4aae1416ad3e48ccd0a4b06ff75f245767195dd66e517e9b72f49bb0a93c3fd7"} May 28 17:13:26.378933 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.378658 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f64ab61a-2865-46ac-822e-8a5d31318958","Type":"ContainerStarted","Data":"99c125e860c7c159eb735cbf7c6f6d57c939d802a6517c4acaa12250e7442bf6"} May 28 17:13:26.378933 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.378671 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f64ab61a-2865-46ac-822e-8a5d31318958","Type":"ContainerStarted","Data":"8f2819ce751b6b563694da2fdfaa4497ad21be799f8c9f5ab860ac5fde0d39a0"} May 28 17:13:26.381428 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.381407 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"32dc81a7-55bb-4586-84f7-c895928c6f26","Type":"ContainerStarted","Data":"2559cbcd88da17ef7ec210e7fe2585f9c21e04fe9d980d66b86f5f40683e5e4b"} May 28 17:13:26.381520 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.381434 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"32dc81a7-55bb-4586-84f7-c895928c6f26","Type":"ContainerStarted","Data":"575744fa03869abf0ae2880ab4462578508ad1f27c1b18cae27c5fe4aa784bf9"} May 28 17:13:26.381520 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.381443 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"32dc81a7-55bb-4586-84f7-c895928c6f26","Type":"ContainerStarted","Data":"3ef245799fb280e456dd552bbd07f79f13c16fc519f5fa9aefc9dab05c593958"} May 28 17:13:26.381520 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.381453 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"32dc81a7-55bb-4586-84f7-c895928c6f26","Type":"ContainerStarted","Data":"bdd98610a32b3a0f0ec95a53eaaa2520370f2f6ba8d5df072f8789d8137b3000"} May 28 17:13:26.381520 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.381461 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"32dc81a7-55bb-4586-84f7-c895928c6f26","Type":"ContainerStarted","Data":"a803d371e015b9976fef09ad8729969722c2e1a654bc4c0e906d15b7a34d275a"} May 28 17:13:26.381520 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.381469 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"32dc81a7-55bb-4586-84f7-c895928c6f26","Type":"ContainerStarted","Data":"443169c331c8230fef48ed97e0eb52c9989aa3882fd4bd54ffd42602b487c6b2"} May 28 17:13:26.382481 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.382460 2566 generic.go:358] "Generic (PLEG): container finished" podID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" containerID="f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86" exitCode=0 May 28 17:13:26.382574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.382510 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" May 28 17:13:26.382574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.382532 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" event={"ID":"86aa82f8-5676-47b2-8cb2-03731a76fb6a","Type":"ContainerDied","Data":"f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86"} May 28 17:13:26.382574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.382557 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b796f6cf6-n4trr" event={"ID":"86aa82f8-5676-47b2-8cb2-03731a76fb6a","Type":"ContainerDied","Data":"a09d0cf7a17745ed2b59d958ae199bc813c12cf74c263aa787f9ee2352a4d3d9"} May 28 17:13:26.382574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.382571 2566 scope.go:117] "RemoveContainer" containerID="f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86" May 28 17:13:26.385253 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.385231 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" event={"ID":"2682f774-cded-415f-a4a3-bafb74e0b6b3","Type":"ContainerStarted","Data":"5d72b638d40a35d8396cdffc4ed7f29507fa2753f438838b6b8be7d5feda0c73"} May 28 17:13:26.385636 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.385598 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:26.391246 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.391228 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" May 28 17:13:26.396872 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.396859 2566 scope.go:117] "RemoveContainer" containerID="f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86" May 28 17:13:26.397123 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:13:26.397106 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86\": container with ID starting with f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86 not found: ID does not exist" containerID="f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86" May 28 17:13:26.397176 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.397129 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86"} err="failed to get container status \"f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86\": rpc error: code = NotFound desc = could not find container \"f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86\": container with ID starting with f6c291d913b9410e81c2de0246e32b23c094cf7b7e8c9a36aeb49222d5b45c86 not found: ID does not exist" May 28 17:13:26.405486 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.405449 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.829065508 podStartE2EDuration="12.40543831s" podCreationTimestamp="2026-05-28 17:13:14 +0000 UTC" firstStartedPulling="2026-05-28 17:13:16.903119852 +0000 UTC m=+206.815512483" lastFinishedPulling="2026-05-28 17:13:23.479492651 +0000 UTC m=+213.391885285" observedRunningTime="2026-05-28 17:13:26.404506269 +0000 UTC m=+216.316898922" watchObservedRunningTime="2026-05-28 17:13:26.40543831 +0000 UTC m=+216.317830963" May 28 17:13:26.423444 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.423414 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b796f6cf6-n4trr"] May 28 17:13:26.427595 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.427570 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b796f6cf6-n4trr"] May 28 17:13:26.451882 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.451799 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b9cd59bcd-z848k" podStartSLOduration=4.855388478 podStartE2EDuration="11.451784648s" podCreationTimestamp="2026-05-28 17:13:15 +0000 UTC" firstStartedPulling="2026-05-28 17:13:16.885161792 +0000 UTC m=+206.797554423" lastFinishedPulling="2026-05-28 17:13:23.481557952 +0000 UTC m=+213.393950593" observedRunningTime="2026-05-28 17:13:26.451108929 +0000 UTC m=+216.363501583" watchObservedRunningTime="2026-05-28 17:13:26.451784648 +0000 UTC m=+216.364177347" May 28 17:13:26.460800 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.460777 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86aa82f8-5676-47b2-8cb2-03731a76fb6a-ca-trust-extracted\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:13:26.460800 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.460801 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-tls\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:13:26.460937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.460811 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-registry-certificates\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:13:26.460937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.460819 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-bound-sa-token\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:13:26.460937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.460828 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-installation-pull-secrets\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:13:26.460937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.460838 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86aa82f8-5676-47b2-8cb2-03731a76fb6a-image-registry-private-configuration\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:13:26.460937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.460847 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4nbs\" (UniqueName: \"kubernetes.io/projected/86aa82f8-5676-47b2-8cb2-03731a76fb6a-kube-api-access-w4nbs\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:13:26.460937 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.460855 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86aa82f8-5676-47b2-8cb2-03731a76fb6a-trusted-ca\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:13:26.482073 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.480425 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.50582241 podStartE2EDuration="7.480406563s" podCreationTimestamp="2026-05-28 17:13:19 +0000 UTC" firstStartedPulling="2026-05-28 17:13:20.329890615 +0000 UTC m=+210.242283251" lastFinishedPulling="2026-05-28 17:13:25.304474756 +0000 UTC m=+215.216867404" observedRunningTime="2026-05-28 17:13:26.478020938 +0000 UTC m=+216.390413593" watchObservedRunningTime="2026-05-28 17:13:26.480406563 +0000 UTC m=+216.392799216" May 28 17:13:26.641838 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:26.641808 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" path="/var/lib/kubelet/pods/86aa82f8-5676-47b2-8cb2-03731a76fb6a/volumes" May 28 17:13:29.704495 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:29.704461 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 28 17:13:32.958605 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:32.958578 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4frtv_37b8dbdd-d75f-47e5-8776-fbb29a0f2104/dns-node-resolver/0.log" May 28 17:13:36.422172 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:36.422134 2566 generic.go:358] "Generic (PLEG): container finished" podID="b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d" containerID="4016fce2fee01b407be872678d7b778a76b0a8c80735103118f3e57c4fcd8f21" exitCode=0 May 28 17:13:36.422682 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:36.422188 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" event={"ID":"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d","Type":"ContainerDied","Data":"4016fce2fee01b407be872678d7b778a76b0a8c80735103118f3e57c4fcd8f21"} May 28 17:13:36.422682 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:36.422589 2566 scope.go:117] "RemoveContainer" containerID="4016fce2fee01b407be872678d7b778a76b0a8c80735103118f3e57c4fcd8f21" May 28 17:13:37.426227 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:37.426194 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-nm9j9" event={"ID":"b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d","Type":"ContainerStarted","Data":"20ae1e8ec4d9827403ebcd591520702af89bc7944d9a7742b70d17fa0ec3e89a"} May 28 17:13:37.928259 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:37.928228 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:37.928418 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:37.928271 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:51.474971 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:51.474865 2566 generic.go:358] "Generic (PLEG): container finished" podID="907ac90b-550c-4975-b881-6e7398ee2437" containerID="523de421b6d74614de433fe92e0cd969eac133e060198e4994887f1895f47de9" exitCode=0 May 28 17:13:51.474971 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:51.474953 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" event={"ID":"907ac90b-550c-4975-b881-6e7398ee2437","Type":"ContainerDied","Data":"523de421b6d74614de433fe92e0cd969eac133e060198e4994887f1895f47de9"} May 28 17:13:51.475385 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:51.475246 2566 scope.go:117] "RemoveContainer" containerID="523de421b6d74614de433fe92e0cd969eac133e060198e4994887f1895f47de9" May 28 17:13:52.479965 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:52.479908 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-82mkf" event={"ID":"907ac90b-550c-4975-b881-6e7398ee2437","Type":"ContainerStarted","Data":"f858690196cca5c8b15450677674d93c133d8e0c45d8e47ce8763aa4c298a121"} May 28 17:13:57.933954 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:57.933892 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:13:57.938044 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:13:57.938023 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56569f4c5-htn9m" May 28 17:14:01.500179 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:01.500140 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:14:01.502372 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:01.502347 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e04539f3-03d0-41ac-8c13-0f67edc8e25c-metrics-certs\") pod \"network-metrics-daemon-lsl89\" (UID: \"e04539f3-03d0-41ac-8c13-0f67edc8e25c\") " pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:14:01.740476 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:01.740449 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t7ndh\"" May 28 17:14:01.748770 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:01.748749 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lsl89" May 28 17:14:01.869475 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:01.869445 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lsl89"] May 28 17:14:01.872663 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:14:01.872633 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04539f3_03d0_41ac_8c13_0f67edc8e25c.slice/crio-e123cfb4eb0b7da5cb0a98968f779329d6e433a8e5b112b08ca78c2bc4b64242 WatchSource:0}: Error finding container e123cfb4eb0b7da5cb0a98968f779329d6e433a8e5b112b08ca78c2bc4b64242: Status 404 returned error can't find the container with id e123cfb4eb0b7da5cb0a98968f779329d6e433a8e5b112b08ca78c2bc4b64242 May 28 17:14:02.513274 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:02.513228 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lsl89" event={"ID":"e04539f3-03d0-41ac-8c13-0f67edc8e25c","Type":"ContainerStarted","Data":"e123cfb4eb0b7da5cb0a98968f779329d6e433a8e5b112b08ca78c2bc4b64242"} May 28 17:14:03.517331 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:03.517298 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lsl89" event={"ID":"e04539f3-03d0-41ac-8c13-0f67edc8e25c","Type":"ContainerStarted","Data":"73d7531582cc4c5442a8df2866b6eebe0ecbf01eb8a7ef43b08f59dcfe1cbeef"} May 28 17:14:03.517331 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:03.517332 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lsl89" event={"ID":"e04539f3-03d0-41ac-8c13-0f67edc8e25c","Type":"ContainerStarted","Data":"48b37be04f43822d71221cea9184e9e6a79f1e72e51f3dd2d4dacacabb64d220"} May 28 17:14:03.536250 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:03.536194 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lsl89" podStartSLOduration=252.212933837 podStartE2EDuration="4m13.536176644s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:14:01.874520153 +0000 UTC m=+251.786912795" lastFinishedPulling="2026-05-28 17:14:03.197762957 +0000 UTC m=+253.110155602" observedRunningTime="2026-05-28 17:14:03.534279163 +0000 UTC m=+253.446671817" watchObservedRunningTime="2026-05-28 17:14:03.536176644 +0000 UTC m=+253.448569298" May 28 17:14:19.704527 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:19.704472 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 28 17:14:19.720983 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:19.720930 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 28 17:14:20.584570 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:20.584544 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 28 17:14:31.162148 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:14:31.162108 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-rqq5s" podUID="9e5b6585-37d1-4e39-8599-509b085c834a" May 28 17:14:31.601467 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:31.601437 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:14:34.385324 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.385286 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:14:34.385810 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.385355 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:14:34.387709 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.387688 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c-metrics-tls\") pod \"dns-default-h68jj\" (UID: \"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c\") " pod="openshift-dns/dns-default-h68jj" May 28 17:14:34.387780 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.387725 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5b6585-37d1-4e39-8599-509b085c834a-cert\") pod \"ingress-canary-rqq5s\" (UID: \"9e5b6585-37d1-4e39-8599-509b085c834a\") " pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:14:34.440152 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.440120 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmnj9\"" May 28 17:14:34.448222 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.448202 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h68jj" May 28 17:14:34.575708 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.575678 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h68jj"] May 28 17:14:34.578031 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:14:34.577995 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e7bbc9_16f7_4cbb_a3b4_c90b14a9428c.slice/crio-d764196c0e070ab4b64b3e0c558a7557b4870c307e73e7c5e86dda9144c88415 WatchSource:0}: Error finding container d764196c0e070ab4b64b3e0c558a7557b4870c307e73e7c5e86dda9144c88415: Status 404 returned error can't find the container with id d764196c0e070ab4b64b3e0c558a7557b4870c307e73e7c5e86dda9144c88415 May 28 17:14:34.605026 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.604999 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jgwcl\"" May 28 17:14:34.611132 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.611106 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h68jj" event={"ID":"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c","Type":"ContainerStarted","Data":"d764196c0e070ab4b64b3e0c558a7557b4870c307e73e7c5e86dda9144c88415"} May 28 17:14:34.613177 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.613164 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rqq5s" May 28 17:14:34.736295 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:34.736153 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rqq5s"] May 28 17:14:34.738769 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:14:34.738746 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5b6585_37d1_4e39_8599_509b085c834a.slice/crio-1ba6c7e1678d5a640b389e77d24c8ffd09dd0a777657ad8970cec2707399219e WatchSource:0}: Error finding container 1ba6c7e1678d5a640b389e77d24c8ffd09dd0a777657ad8970cec2707399219e: Status 404 returned error can't find the container with id 1ba6c7e1678d5a640b389e77d24c8ffd09dd0a777657ad8970cec2707399219e May 28 17:14:35.616205 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:35.616159 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rqq5s" event={"ID":"9e5b6585-37d1-4e39-8599-509b085c834a","Type":"ContainerStarted","Data":"1ba6c7e1678d5a640b389e77d24c8ffd09dd0a777657ad8970cec2707399219e"} May 28 17:14:36.621299 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:36.621272 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h68jj" event={"ID":"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c","Type":"ContainerStarted","Data":"d53578b13a2e05e2090dfebe30cc50c13dec49d708d7856c51486389313e4e0b"} May 28 17:14:36.622616 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:36.622497 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rqq5s" event={"ID":"9e5b6585-37d1-4e39-8599-509b085c834a","Type":"ContainerStarted","Data":"40653bed1dc61d36ebce513e7ab2b2f287f9d09132d7b7ca59f627da7a193c2c"} May 28 17:14:36.639299 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:36.639254 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rqq5s" podStartSLOduration=250.961343914 podStartE2EDuration="4m12.639237181s" podCreationTimestamp="2026-05-28 17:10:24 +0000 UTC" firstStartedPulling="2026-05-28 17:14:34.740551799 +0000 UTC m=+284.652944430" lastFinishedPulling="2026-05-28 17:14:36.418445067 +0000 UTC m=+286.330837697" observedRunningTime="2026-05-28 17:14:36.637699073 +0000 UTC m=+286.550091727" watchObservedRunningTime="2026-05-28 17:14:36.639237181 +0000 UTC m=+286.551629835" May 28 17:14:37.626879 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:37.626837 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h68jj" event={"ID":"b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c","Type":"ContainerStarted","Data":"ff131a3a057f84efd9690c527ec8cf40d95da02009edd1a1f6c11550708b60bb"} May 28 17:14:37.627288 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:37.626964 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h68jj" May 28 17:14:37.644618 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:37.644571 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-h68jj" podStartSLOduration=251.808007685 podStartE2EDuration="4m13.644559223s" podCreationTimestamp="2026-05-28 17:10:24 +0000 UTC" firstStartedPulling="2026-05-28 17:14:34.579694432 +0000 UTC m=+284.492087063" lastFinishedPulling="2026-05-28 17:14:36.416245966 +0000 UTC m=+286.328638601" observedRunningTime="2026-05-28 17:14:37.643430875 +0000 UTC m=+287.555823529" watchObservedRunningTime="2026-05-28 17:14:37.644559223 +0000 UTC m=+287.556951876" May 28 17:14:47.632305 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:47.632270 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h68jj" May 28 17:14:50.541836 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:14:50.541813 2566 kubelet.go:1628] "Image garbage collection succeeded" May 28 17:16:57.283134 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.283097 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt"] May 28 17:16:57.283565 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.283427 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" containerName="registry" May 28 17:16:57.283565 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.283437 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" containerName="registry" May 28 17:16:57.283565 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.283485 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="86aa82f8-5676-47b2-8cb2-03731a76fb6a" containerName="registry" May 28 17:16:57.289524 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.289495 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.295158 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.295135 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" May 28 17:16:57.296316 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.296270 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" May 28 17:16:57.296401 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.296374 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" May 28 17:16:57.296705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.296506 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2cv69\"" May 28 17:16:57.296705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.296618 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" May 28 17:16:57.299484 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.299464 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" May 28 17:16:57.301072 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.301055 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt"] May 28 17:16:57.353058 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.353030 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw"] May 28 17:16:57.355559 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.355536 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.360247 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.360226 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 28 17:16:57.362117 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.362096 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 28 17:16:57.362117 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.362117 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 28 17:16:57.362262 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.362132 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s4wrb\"" May 28 17:16:57.364755 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.364739 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 28 17:16:57.387738 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.387717 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw"] May 28 17:16:57.405493 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.405471 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20dd6b96-7793-4fa7-94e8-08c44febd7fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.405589 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.405518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20dd6b96-7793-4fa7-94e8-08c44febd7fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.405589 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.405552 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-metrics-cert\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.405589 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.405577 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-manager-config\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.405701 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.405616 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhmsg\" (UniqueName: \"kubernetes.io/projected/20dd6b96-7793-4fa7-94e8-08c44febd7fb-kube-api-access-bhmsg\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.405701 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.405634 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-cert\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.405701 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.405654 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbstd\" (UniqueName: \"kubernetes.io/projected/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-kube-api-access-pbstd\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.507059 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.507028 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20dd6b96-7793-4fa7-94e8-08c44febd7fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.507251 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.507076 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-metrics-cert\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.507251 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.507116 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-manager-config\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.507251 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.507164 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bhmsg\" (UniqueName: \"kubernetes.io/projected/20dd6b96-7793-4fa7-94e8-08c44febd7fb-kube-api-access-bhmsg\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.507251 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.507202 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-cert\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.507251 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.507236 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pbstd\" (UniqueName: \"kubernetes.io/projected/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-kube-api-access-pbstd\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.507505 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.507283 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20dd6b96-7793-4fa7-94e8-08c44febd7fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.507928 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.507905 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-manager-config\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.509586 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.509566 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20dd6b96-7793-4fa7-94e8-08c44febd7fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.509681 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.509601 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20dd6b96-7793-4fa7-94e8-08c44febd7fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.509981 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.509961 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-metrics-cert\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.510043 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.510001 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-cert\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.527619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.527598 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbstd\" (UniqueName: \"kubernetes.io/projected/ce322ec3-5cb5-4e80-8bc4-14ca438f06ed-kube-api-access-pbstd\") pod \"lws-controller-manager-7dc4c66bd5-lshzt\" (UID: \"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed\") " pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.527711 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.527602 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhmsg\" (UniqueName: \"kubernetes.io/projected/20dd6b96-7793-4fa7-94e8-08c44febd7fb-kube-api-access-bhmsg\") pod \"opendatahub-operator-controller-manager-64688695c6-c6bkw\" (UID: \"20dd6b96-7793-4fa7-94e8-08c44febd7fb\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.599649 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.599582 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:16:57.665740 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.665708 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:16:57.768234 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.768209 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt"] May 28 17:16:57.770848 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:16:57.770815 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce322ec3_5cb5_4e80_8bc4_14ca438f06ed.slice/crio-3dc6674789a297ce8a7a6ba37d8b8739eb72d54583152b3fdf1be15cf3244e90 WatchSource:0}: Error finding container 3dc6674789a297ce8a7a6ba37d8b8739eb72d54583152b3fdf1be15cf3244e90: Status 404 returned error can't find the container with id 3dc6674789a297ce8a7a6ba37d8b8739eb72d54583152b3fdf1be15cf3244e90 May 28 17:16:57.772757 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:57.772741 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 17:16:58.030039 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:58.029998 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw"] May 28 17:16:58.034686 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:16:58.034657 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20dd6b96_7793_4fa7_94e8_08c44febd7fb.slice/crio-dd3f9e629b5b31e692656e0e2a0c7597c7f4d9431eb368d07d7b1863563e7dad WatchSource:0}: Error finding container dd3f9e629b5b31e692656e0e2a0c7597c7f4d9431eb368d07d7b1863563e7dad: Status 404 returned error can't find the container with id dd3f9e629b5b31e692656e0e2a0c7597c7f4d9431eb368d07d7b1863563e7dad May 28 17:16:58.043586 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:58.043558 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" event={"ID":"20dd6b96-7793-4fa7-94e8-08c44febd7fb","Type":"ContainerStarted","Data":"dd3f9e629b5b31e692656e0e2a0c7597c7f4d9431eb368d07d7b1863563e7dad"} May 28 17:16:58.044525 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:16:58.044504 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" event={"ID":"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed","Type":"ContainerStarted","Data":"3dc6674789a297ce8a7a6ba37d8b8739eb72d54583152b3fdf1be15cf3244e90"} May 28 17:17:02.059582 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:02.059535 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" event={"ID":"20dd6b96-7793-4fa7-94e8-08c44febd7fb","Type":"ContainerStarted","Data":"5dae7d9bb55e94078767371692f70743e7b09bb0b61a3e849b346abf371560f3"} May 28 17:17:02.060071 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:02.059660 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:17:02.060939 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:02.060915 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" event={"ID":"ce322ec3-5cb5-4e80-8bc4-14ca438f06ed","Type":"ContainerStarted","Data":"e8b0a0bb3e831c49681b221d66ecb9a478a3f297c7ec3c15c9cf7ece7cf76537"} May 28 17:17:02.061091 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:02.061028 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:17:02.085362 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:02.085313 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" podStartSLOduration=1.512300051 podStartE2EDuration="5.08530126s" podCreationTimestamp="2026-05-28 17:16:57 +0000 UTC" firstStartedPulling="2026-05-28 17:16:58.038602572 +0000 UTC m=+427.950995203" lastFinishedPulling="2026-05-28 17:17:01.611603767 +0000 UTC m=+431.523996412" observedRunningTime="2026-05-28 17:17:02.084537553 +0000 UTC m=+431.996930207" watchObservedRunningTime="2026-05-28 17:17:02.08530126 +0000 UTC m=+431.997693982" May 28 17:17:02.103010 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:02.102971 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" podStartSLOduration=1.203504082 podStartE2EDuration="5.102926028s" podCreationTimestamp="2026-05-28 17:16:57 +0000 UTC" firstStartedPulling="2026-05-28 17:16:57.77289497 +0000 UTC m=+427.685287601" lastFinishedPulling="2026-05-28 17:17:01.672316913 +0000 UTC m=+431.584709547" observedRunningTime="2026-05-28 17:17:02.101007271 +0000 UTC m=+432.013399924" watchObservedRunningTime="2026-05-28 17:17:02.102926028 +0000 UTC m=+432.015318686" May 28 17:17:13.066856 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:13.066778 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7dc4c66bd5-lshzt" May 28 17:17:13.066856 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:13.066828 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-c6bkw" May 28 17:17:44.858625 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.858593 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l"] May 28 17:17:44.861150 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.861129 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.863781 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.863757 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kwfzv\"" May 28 17:17:44.863887 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.863765 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 28 17:17:44.875910 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.875884 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l"] May 28 17:17:44.908211 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908190 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61354755-75b9-468a-ac38-042903b1b177-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.908314 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61354755-75b9-468a-ac38-042903b1b177-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.908314 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.908396 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.908396 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908363 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.908468 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m7jz\" (UniqueName: \"kubernetes.io/projected/61354755-75b9-468a-ac38-042903b1b177-kube-api-access-9m7jz\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.908468 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61354755-75b9-468a-ac38-042903b1b177-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.908468 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908450 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:44.908568 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:44.908481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009497 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009463 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009497 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009500 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9m7jz\" (UniqueName: \"kubernetes.io/projected/61354755-75b9-468a-ac38-042903b1b177-kube-api-access-9m7jz\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009666 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009523 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61354755-75b9-468a-ac38-042903b1b177-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009666 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009548 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009666 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009572 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009666 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009606 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61354755-75b9-468a-ac38-042903b1b177-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009666 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009651 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61354755-75b9-468a-ac38-042903b1b177-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009854 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009699 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009854 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009719 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.009919 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.009885 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.010114 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.010092 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.010180 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.010160 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.010228 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.010190 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.010446 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.010428 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61354755-75b9-468a-ac38-042903b1b177-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.011935 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.011914 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61354755-75b9-468a-ac38-042903b1b177-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.012103 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.012085 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61354755-75b9-468a-ac38-042903b1b177-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.018235 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.018207 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61354755-75b9-468a-ac38-042903b1b177-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.018670 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.018649 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m7jz\" (UniqueName: \"kubernetes.io/projected/61354755-75b9-468a-ac38-042903b1b177-kube-api-access-9m7jz\") pod \"data-science-gateway-data-science-gateway-class-595b7776f86zh8l\" (UID: \"61354755-75b9-468a-ac38-042903b1b177\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.171859 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.171782 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:45.297032 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:45.297007 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l"] May 28 17:17:46.209817 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:46.209774 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" event={"ID":"61354755-75b9-468a-ac38-042903b1b177","Type":"ContainerStarted","Data":"b6d8d6ed78327809d1db9c6138a3cb6d6276ca24350c5502235040520f56ce9b"} May 28 17:17:47.936779 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:47.936744 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} May 28 17:17:47.937031 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:47.936817 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} May 28 17:17:47.937031 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:47.936845 2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} May 28 17:17:48.217171 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:48.217083 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" event={"ID":"61354755-75b9-468a-ac38-042903b1b177","Type":"ContainerStarted","Data":"bd78ec298a95775bc6936c5214ce4508ef163722dfba157856bd74e726415f98"} May 28 17:17:48.236783 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:48.236738 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" podStartSLOduration=1.6035010060000001 podStartE2EDuration="4.236721219s" podCreationTimestamp="2026-05-28 17:17:44 +0000 UTC" firstStartedPulling="2026-05-28 17:17:45.303316137 +0000 UTC m=+475.215708768" lastFinishedPulling="2026-05-28 17:17:47.936536341 +0000 UTC m=+477.848928981" observedRunningTime="2026-05-28 17:17:48.235188052 +0000 UTC m=+478.147580704" watchObservedRunningTime="2026-05-28 17:17:48.236721219 +0000 UTC m=+478.149113873" May 28 17:17:49.172618 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:49.172586 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:49.177258 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:49.177234 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:49.220670 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:49.220642 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:17:49.221481 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:17:49.221462 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f86zh8l" May 28 17:18:16.238635 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.238603 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bzx78"] May 28 17:18:16.244211 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.244193 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" May 28 17:18:16.246922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.246902 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 28 17:18:16.247903 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.247883 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lp2dq\"" May 28 17:18:16.248135 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.248117 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 28 17:18:16.250180 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.250156 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bzx78"] May 28 17:18:16.386615 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.386586 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-597lw\" (UniqueName: \"kubernetes.io/projected/80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7-kube-api-access-597lw\") pod \"kuadrant-operator-catalog-bzx78\" (UID: \"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7\") " pod="kuadrant-system/kuadrant-operator-catalog-bzx78" May 28 17:18:16.488078 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.488042 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-597lw\" (UniqueName: \"kubernetes.io/projected/80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7-kube-api-access-597lw\") pod \"kuadrant-operator-catalog-bzx78\" (UID: \"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7\") " pod="kuadrant-system/kuadrant-operator-catalog-bzx78" May 28 17:18:16.497054 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.497032 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-597lw\" (UniqueName: \"kubernetes.io/projected/80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7-kube-api-access-597lw\") pod \"kuadrant-operator-catalog-bzx78\" (UID: \"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7\") " pod="kuadrant-system/kuadrant-operator-catalog-bzx78" May 28 17:18:16.554470 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.554441 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" May 28 17:18:16.561015 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.560992 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bzx78"] May 28 17:18:16.680457 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.680433 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bzx78"] May 28 17:18:16.683183 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:18:16.683155 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f69764_9eb9_4bf9_a4f8_d33eaf59d0b7.slice/crio-7567b1346be9336e01fb8d66cf77d973a420946660303bfff6637396d6fb3c36 WatchSource:0}: Error finding container 7567b1346be9336e01fb8d66cf77d973a420946660303bfff6637396d6fb3c36: Status 404 returned error can't find the container with id 7567b1346be9336e01fb8d66cf77d973a420946660303bfff6637396d6fb3c36 May 28 17:18:16.775749 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.775666 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldp46"] May 28 17:18:16.780023 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.780008 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:16.785635 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.785611 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldp46"] May 28 17:18:16.890496 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.890454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svxzv\" (UniqueName: \"kubernetes.io/projected/09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0-kube-api-access-svxzv\") pod \"kuadrant-operator-catalog-ldp46\" (UID: \"09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0\") " pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:16.991533 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:16.991490 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-svxzv\" (UniqueName: \"kubernetes.io/projected/09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0-kube-api-access-svxzv\") pod \"kuadrant-operator-catalog-ldp46\" (UID: \"09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0\") " pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:17.000080 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:17.000059 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svxzv\" (UniqueName: \"kubernetes.io/projected/09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0-kube-api-access-svxzv\") pod \"kuadrant-operator-catalog-ldp46\" (UID: \"09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0\") " pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:17.089982 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:17.089894 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:17.243588 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:17.243559 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldp46"] May 28 17:18:17.249499 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:18:17.249470 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ac591a_bf75_4b7c_ba74_54ae1f0aa6b0.slice/crio-60fa866b08f8b1527e0f56b1bd39543fe4f6ee500c42e2bbf0cfe9032bc9f043 WatchSource:0}: Error finding container 60fa866b08f8b1527e0f56b1bd39543fe4f6ee500c42e2bbf0cfe9032bc9f043: Status 404 returned error can't find the container with id 60fa866b08f8b1527e0f56b1bd39543fe4f6ee500c42e2bbf0cfe9032bc9f043 May 28 17:18:17.316829 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:17.316788 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" event={"ID":"09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0","Type":"ContainerStarted","Data":"60fa866b08f8b1527e0f56b1bd39543fe4f6ee500c42e2bbf0cfe9032bc9f043"} May 28 17:18:17.318012 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:17.317989 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" event={"ID":"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7","Type":"ContainerStarted","Data":"7567b1346be9336e01fb8d66cf77d973a420946660303bfff6637396d6fb3c36"} May 28 17:18:19.328081 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.328044 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" event={"ID":"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7","Type":"ContainerStarted","Data":"14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d"} May 28 17:18:19.328504 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.328117 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" podUID="80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7" containerName="registry-server" containerID="cri-o://14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d" gracePeriod=2 May 28 17:18:19.329490 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.329468 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" event={"ID":"09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0","Type":"ContainerStarted","Data":"3a6f266162f9c0f43b4881b5f45e50a035c1d92c458f453f38f003ccd0874c30"} May 28 17:18:19.343675 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.343628 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" podStartSLOduration=1.082391966 podStartE2EDuration="3.343611614s" podCreationTimestamp="2026-05-28 17:18:16 +0000 UTC" firstStartedPulling="2026-05-28 17:18:16.684496549 +0000 UTC m=+506.596889180" lastFinishedPulling="2026-05-28 17:18:18.94571598 +0000 UTC m=+508.858108828" observedRunningTime="2026-05-28 17:18:19.342554552 +0000 UTC m=+509.254947205" watchObservedRunningTime="2026-05-28 17:18:19.343611614 +0000 UTC m=+509.256004269" May 28 17:18:19.359443 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.359391 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" podStartSLOduration=1.6607285379999999 podStartE2EDuration="3.359379845s" podCreationTimestamp="2026-05-28 17:18:16 +0000 UTC" firstStartedPulling="2026-05-28 17:18:17.250803126 +0000 UTC m=+507.163195756" lastFinishedPulling="2026-05-28 17:18:18.949454431 +0000 UTC m=+508.861847063" observedRunningTime="2026-05-28 17:18:19.35835678 +0000 UTC m=+509.270749432" watchObservedRunningTime="2026-05-28 17:18:19.359379845 +0000 UTC m=+509.271772499" May 28 17:18:19.571390 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.571366 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" May 28 17:18:19.617241 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.617172 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-597lw\" (UniqueName: \"kubernetes.io/projected/80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7-kube-api-access-597lw\") pod \"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7\" (UID: \"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7\") " May 28 17:18:19.619254 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.619235 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7-kube-api-access-597lw" (OuterVolumeSpecName: "kube-api-access-597lw") pod "80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7" (UID: "80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7"). InnerVolumeSpecName "kube-api-access-597lw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 17:18:19.718307 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:19.718274 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-597lw\" (UniqueName: \"kubernetes.io/projected/80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7-kube-api-access-597lw\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:18:20.333674 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.333624 2566 generic.go:358] "Generic (PLEG): container finished" podID="80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7" containerID="14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d" exitCode=0 May 28 17:18:20.334167 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.333692 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" May 28 17:18:20.334167 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.333707 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" event={"ID":"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7","Type":"ContainerDied","Data":"14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d"} May 28 17:18:20.334167 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.333749 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bzx78" event={"ID":"80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7","Type":"ContainerDied","Data":"7567b1346be9336e01fb8d66cf77d973a420946660303bfff6637396d6fb3c36"} May 28 17:18:20.334167 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.333766 2566 scope.go:117] "RemoveContainer" containerID="14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d" May 28 17:18:20.342714 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.342679 2566 scope.go:117] "RemoveContainer" containerID="14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d" May 28 17:18:20.342973 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:18:20.342932 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d\": container with ID starting with 14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d not found: ID does not exist" containerID="14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d" May 28 17:18:20.343035 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.342982 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d"} err="failed to get container status \"14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d\": rpc error: code = NotFound desc = could not find container \"14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d\": container with ID starting with 14f921839d7f49f0b6065aaef05032702ef7058c6a14e2bbd82bd1d5f639e46d not found: ID does not exist" May 28 17:18:20.355596 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.355571 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bzx78"] May 28 17:18:20.359058 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.359036 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bzx78"] May 28 17:18:20.642427 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:20.642357 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7" path="/var/lib/kubelet/pods/80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7/volumes" May 28 17:18:27.090711 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:27.090674 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:27.091260 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:27.090758 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:27.112721 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:27.112692 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:27.378462 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:27.378393 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-ldp46" May 28 17:18:48.541999 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.541904 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-s6qmg"] May 28 17:18:48.542447 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.542285 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7" containerName="registry-server" May 28 17:18:48.542447 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.542299 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7" containerName="registry-server" May 28 17:18:48.542447 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.542348 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="80f69764-9eb9-4bf9-a4f8-d33eaf59d0b7" containerName="registry-server" May 28 17:18:48.548742 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.548725 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" May 28 17:18:48.553109 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.553092 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-96lxh\"" May 28 17:18:48.557297 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.557274 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-s6qmg"] May 28 17:18:48.665269 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.665244 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp6bk\" (UniqueName: \"kubernetes.io/projected/a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3-kube-api-access-fp6bk\") pod \"authorino-operator-657f44b778-s6qmg\" (UID: \"a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3\") " pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" May 28 17:18:48.766119 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.766090 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fp6bk\" (UniqueName: \"kubernetes.io/projected/a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3-kube-api-access-fp6bk\") pod \"authorino-operator-657f44b778-s6qmg\" (UID: \"a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3\") " pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" May 28 17:18:48.775851 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.775824 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp6bk\" (UniqueName: \"kubernetes.io/projected/a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3-kube-api-access-fp6bk\") pod \"authorino-operator-657f44b778-s6qmg\" (UID: \"a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3\") " pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" May 28 17:18:48.858911 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.858827 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" May 28 17:18:48.982934 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:48.982906 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-s6qmg"] May 28 17:18:48.985236 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:18:48.985207 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c8cb9a_36d2_4b92_81e8_6f6803ed9cb3.slice/crio-d8d0a43f19980fd08c6b20259b56dc5918eb32320fec6a8e59b4d38d25a1fb87 WatchSource:0}: Error finding container d8d0a43f19980fd08c6b20259b56dc5918eb32320fec6a8e59b4d38d25a1fb87: Status 404 returned error can't find the container with id d8d0a43f19980fd08c6b20259b56dc5918eb32320fec6a8e59b4d38d25a1fb87 May 28 17:18:49.427014 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:49.426978 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" event={"ID":"a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3","Type":"ContainerStarted","Data":"d8d0a43f19980fd08c6b20259b56dc5918eb32320fec6a8e59b4d38d25a1fb87"} May 28 17:18:51.435511 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:51.435472 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" event={"ID":"a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3","Type":"ContainerStarted","Data":"7b03286ff8507245062fd97f36fe9b73522c98399c52e254ca0c6d4d7b954447"} May 28 17:18:51.436022 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:51.435628 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" May 28 17:18:51.453382 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:51.453322 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" podStartSLOduration=1.154952363 podStartE2EDuration="3.453304906s" podCreationTimestamp="2026-05-28 17:18:48 +0000 UTC" firstStartedPulling="2026-05-28 17:18:48.9870691 +0000 UTC m=+538.899461731" lastFinishedPulling="2026-05-28 17:18:51.285421627 +0000 UTC m=+541.197814274" observedRunningTime="2026-05-28 17:18:51.452561199 +0000 UTC m=+541.364953843" watchObservedRunningTime="2026-05-28 17:18:51.453304906 +0000 UTC m=+541.365697560" May 28 17:18:58.480990 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.480957 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp"] May 28 17:18:58.485688 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.485669 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:18:58.488342 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.488320 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9r5b8\"" May 28 17:18:58.496968 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.496927 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp"] May 28 17:18:58.555065 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.555025 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee9d6e3f-3291-4436-9479-2e6da5a32948-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" (UID: \"ee9d6e3f-3291-4436-9479-2e6da5a32948\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:18:58.555235 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.555143 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j925x\" (UniqueName: \"kubernetes.io/projected/ee9d6e3f-3291-4436-9479-2e6da5a32948-kube-api-access-j925x\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" (UID: \"ee9d6e3f-3291-4436-9479-2e6da5a32948\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:18:58.656078 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.656039 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee9d6e3f-3291-4436-9479-2e6da5a32948-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" (UID: \"ee9d6e3f-3291-4436-9479-2e6da5a32948\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:18:58.656239 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.656112 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j925x\" (UniqueName: \"kubernetes.io/projected/ee9d6e3f-3291-4436-9479-2e6da5a32948-kube-api-access-j925x\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" (UID: \"ee9d6e3f-3291-4436-9479-2e6da5a32948\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:18:58.656402 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.656382 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee9d6e3f-3291-4436-9479-2e6da5a32948-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" (UID: \"ee9d6e3f-3291-4436-9479-2e6da5a32948\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:18:58.668035 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.668005 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j925x\" (UniqueName: \"kubernetes.io/projected/ee9d6e3f-3291-4436-9479-2e6da5a32948-kube-api-access-j925x\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" (UID: \"ee9d6e3f-3291-4436-9479-2e6da5a32948\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:18:58.795740 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.795704 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:18:58.926328 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:58.926248 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp"] May 28 17:18:58.928960 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:18:58.928916 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9d6e3f_3291_4436_9479_2e6da5a32948.slice/crio-f54fc1230228b6f4a5031a1cb264e6bf326cba6103d0db54bcd8a2a9b58a9b5d WatchSource:0}: Error finding container f54fc1230228b6f4a5031a1cb264e6bf326cba6103d0db54bcd8a2a9b58a9b5d: Status 404 returned error can't find the container with id f54fc1230228b6f4a5031a1cb264e6bf326cba6103d0db54bcd8a2a9b58a9b5d May 28 17:18:59.463297 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:18:59.463264 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" event={"ID":"ee9d6e3f-3291-4436-9479-2e6da5a32948","Type":"ContainerStarted","Data":"f54fc1230228b6f4a5031a1cb264e6bf326cba6103d0db54bcd8a2a9b58a9b5d"} May 28 17:19:02.442400 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:02.442368 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-s6qmg" May 28 17:19:04.481997 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:04.481891 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" event={"ID":"ee9d6e3f-3291-4436-9479-2e6da5a32948","Type":"ContainerStarted","Data":"6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006"} May 28 17:19:04.482365 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:04.482050 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:19:04.502173 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:04.502121 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" podStartSLOduration=1.229486963 podStartE2EDuration="6.502105986s" podCreationTimestamp="2026-05-28 17:18:58 +0000 UTC" firstStartedPulling="2026-05-28 17:18:58.931830525 +0000 UTC m=+548.844223155" lastFinishedPulling="2026-05-28 17:19:04.20444954 +0000 UTC m=+554.116842178" observedRunningTime="2026-05-28 17:19:04.499880413 +0000 UTC m=+554.412273068" watchObservedRunningTime="2026-05-28 17:19:04.502105986 +0000 UTC m=+554.414498639" May 28 17:19:15.488169 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:15.488134 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:19:17.109486 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.109448 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp"] May 28 17:19:17.110020 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.109720 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" containerName="manager" containerID="cri-o://6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006" gracePeriod=2 May 28 17:19:17.115519 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.115491 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp"] May 28 17:19:17.138436 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.138411 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h"] May 28 17:19:17.138894 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.138877 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" containerName="manager" May 28 17:19:17.138894 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.138895 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" containerName="manager" May 28 17:19:17.139056 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.139021 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" containerName="manager" May 28 17:19:17.142042 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.142024 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:17.161392 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.161362 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h"] May 28 17:19:17.179257 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.179219 2566 status_manager.go:895] "Failed to get status for pod" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" is forbidden: User \"system:node:ip-10-0-130-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-46.ec2.internal' and this object" May 28 17:19:17.202783 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.202757 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p"] May 28 17:19:17.206363 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.206347 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" May 28 17:19:17.209425 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.209404 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4p8gd\"" May 28 17:19:17.216734 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.216714 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p"] May 28 17:19:17.225386 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.225361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nz5j\" (UniqueName: \"kubernetes.io/projected/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-kube-api-access-6nz5j\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-9hw7h\" (UID: \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:17.225485 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.225400 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfjrc\" (UniqueName: \"kubernetes.io/projected/4cbf0533-9be5-45ba-a62c-39093f09a8a4-kube-api-access-bfjrc\") pod \"limitador-operator-controller-manager-85c4996f8c-smf2p\" (UID: \"4cbf0533-9be5-45ba-a62c-39093f09a8a4\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" May 28 17:19:17.225550 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.225530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-9hw7h\" (UID: \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:17.230633 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.230604 2566 status_manager.go:895] "Failed to get status for pod" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" is forbidden: User \"system:node:ip-10-0-130-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-46.ec2.internal' and this object" May 28 17:19:17.326842 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.326805 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6nz5j\" (UniqueName: \"kubernetes.io/projected/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-kube-api-access-6nz5j\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-9hw7h\" (UID: \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:17.327017 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.326855 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfjrc\" (UniqueName: \"kubernetes.io/projected/4cbf0533-9be5-45ba-a62c-39093f09a8a4-kube-api-access-bfjrc\") pod \"limitador-operator-controller-manager-85c4996f8c-smf2p\" (UID: \"4cbf0533-9be5-45ba-a62c-39093f09a8a4\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" May 28 17:19:17.327017 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.326969 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-9hw7h\" (UID: \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:17.327363 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.327342 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-9hw7h\" (UID: \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:17.337301 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.337277 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nz5j\" (UniqueName: \"kubernetes.io/projected/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-kube-api-access-6nz5j\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-9hw7h\" (UID: \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:17.337485 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.337462 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfjrc\" (UniqueName: \"kubernetes.io/projected/4cbf0533-9be5-45ba-a62c-39093f09a8a4-kube-api-access-bfjrc\") pod \"limitador-operator-controller-manager-85c4996f8c-smf2p\" (UID: \"4cbf0533-9be5-45ba-a62c-39093f09a8a4\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" May 28 17:19:17.357684 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.357662 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:19:17.359969 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.359870 2566 status_manager.go:895] "Failed to get status for pod" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" is forbidden: User \"system:node:ip-10-0-130-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-46.ec2.internal' and this object" May 28 17:19:17.427647 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.427619 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j925x\" (UniqueName: \"kubernetes.io/projected/ee9d6e3f-3291-4436-9479-2e6da5a32948-kube-api-access-j925x\") pod \"ee9d6e3f-3291-4436-9479-2e6da5a32948\" (UID: \"ee9d6e3f-3291-4436-9479-2e6da5a32948\") " May 28 17:19:17.427813 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.427689 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee9d6e3f-3291-4436-9479-2e6da5a32948-extensions-socket-volume\") pod \"ee9d6e3f-3291-4436-9479-2e6da5a32948\" (UID: \"ee9d6e3f-3291-4436-9479-2e6da5a32948\") " May 28 17:19:17.428237 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.428203 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9d6e3f-3291-4436-9479-2e6da5a32948-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ee9d6e3f-3291-4436-9479-2e6da5a32948" (UID: "ee9d6e3f-3291-4436-9479-2e6da5a32948"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 17:19:17.429668 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.429646 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9d6e3f-3291-4436-9479-2e6da5a32948-kube-api-access-j925x" (OuterVolumeSpecName: "kube-api-access-j925x") pod "ee9d6e3f-3291-4436-9479-2e6da5a32948" (UID: "ee9d6e3f-3291-4436-9479-2e6da5a32948"). InnerVolumeSpecName "kube-api-access-j925x". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 17:19:17.491085 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.491054 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:17.517809 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.517779 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" May 28 17:19:17.528531 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.528485 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee9d6e3f-3291-4436-9479-2e6da5a32948-extensions-socket-volume\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:19:17.528681 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.528538 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j925x\" (UniqueName: \"kubernetes.io/projected/ee9d6e3f-3291-4436-9479-2e6da5a32948-kube-api-access-j925x\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:19:17.532615 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.532583 2566 generic.go:358] "Generic (PLEG): container finished" podID="ee9d6e3f-3291-4436-9479-2e6da5a32948" containerID="6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006" exitCode=0 May 28 17:19:17.532728 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.532633 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" May 28 17:19:17.532728 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.532658 2566 scope.go:117] "RemoveContainer" containerID="6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006" May 28 17:19:17.535237 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.535200 2566 status_manager.go:895] "Failed to get status for pod" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" is forbidden: User \"system:node:ip-10-0-130-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-46.ec2.internal' and this object" May 28 17:19:17.546281 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.545881 2566 status_manager.go:895] "Failed to get status for pod" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" is forbidden: User \"system:node:ip-10-0-130-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-46.ec2.internal' and this object" May 28 17:19:17.546548 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.546407 2566 scope.go:117] "RemoveContainer" containerID="6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006" May 28 17:19:17.546852 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:19:17.546825 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006\": container with ID starting with 6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006 not found: ID does not exist" containerID="6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006" May 28 17:19:17.546965 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.546862 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006"} err="failed to get container status \"6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006\": rpc error: code = NotFound desc = could not find container \"6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006\": container with ID starting with 6f175612a85f09884057c93a3f69b0487718fb3b3cfdc8a89f04c83504c05006 not found: ID does not exist" May 28 17:19:17.639444 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.639415 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h"] May 28 17:19:17.642273 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:19:17.642241 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110e42ac_b57a_4ebc_aaec_24f6d29ec5ba.slice/crio-d2ef081398841f4db31e596726f4015ab971f85bda7735baa3820772f6b8501e WatchSource:0}: Error finding container d2ef081398841f4db31e596726f4015ab971f85bda7735baa3820772f6b8501e: Status 404 returned error can't find the container with id d2ef081398841f4db31e596726f4015ab971f85bda7735baa3820772f6b8501e May 28 17:19:17.660870 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:17.660834 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p"] May 28 17:19:17.662928 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:19:17.662898 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cbf0533_9be5_45ba_a62c_39093f09a8a4.slice/crio-657852b443648b48af270ecdcfbd57643119e54aa0042c1d2f3ba6e4f1fde684 WatchSource:0}: Error finding container 657852b443648b48af270ecdcfbd57643119e54aa0042c1d2f3ba6e4f1fde684: Status 404 returned error can't find the container with id 657852b443648b48af270ecdcfbd57643119e54aa0042c1d2f3ba6e4f1fde684 May 28 17:19:18.539361 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:18.539316 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" event={"ID":"4cbf0533-9be5-45ba-a62c-39093f09a8a4","Type":"ContainerStarted","Data":"657852b443648b48af270ecdcfbd57643119e54aa0042c1d2f3ba6e4f1fde684"} May 28 17:19:18.542140 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:18.542112 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" event={"ID":"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba","Type":"ContainerStarted","Data":"0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866"} May 28 17:19:18.542276 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:18.542145 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" event={"ID":"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba","Type":"ContainerStarted","Data":"d2ef081398841f4db31e596726f4015ab971f85bda7735baa3820772f6b8501e"} May 28 17:19:18.542276 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:18.542271 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:18.581329 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:18.581299 2566 status_manager.go:895] "Failed to get status for pod" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6z6qp" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-6z6qp\" is forbidden: User \"system:node:ip-10-0-130-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-46.ec2.internal' and this object" May 28 17:19:18.581517 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:18.581472 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" podStartSLOduration=1.581455127 podStartE2EDuration="1.581455127s" podCreationTimestamp="2026-05-28 17:19:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:19:18.579248444 +0000 UTC m=+568.491641100" watchObservedRunningTime="2026-05-28 17:19:18.581455127 +0000 UTC m=+568.493847782" May 28 17:19:18.642401 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:18.642372 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9d6e3f-3291-4436-9479-2e6da5a32948" path="/var/lib/kubelet/pods/ee9d6e3f-3291-4436-9479-2e6da5a32948/volumes" May 28 17:19:19.546859 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:19.546821 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" event={"ID":"4cbf0533-9be5-45ba-a62c-39093f09a8a4","Type":"ContainerStarted","Data":"1765a5068a61f92a93d30e6455c694d99985423648e2632dcaad9b8aaa225b30"} May 28 17:19:19.564146 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:19.564093 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" podStartSLOduration=1.168387307 podStartE2EDuration="2.564079898s" podCreationTimestamp="2026-05-28 17:19:17 +0000 UTC" firstStartedPulling="2026-05-28 17:19:17.664870613 +0000 UTC m=+567.577263244" lastFinishedPulling="2026-05-28 17:19:19.060563186 +0000 UTC m=+568.972955835" observedRunningTime="2026-05-28 17:19:19.563033875 +0000 UTC m=+569.475426538" watchObservedRunningTime="2026-05-28 17:19:19.564079898 +0000 UTC m=+569.476472550" May 28 17:19:20.550621 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:20.550590 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" May 28 17:19:29.549397 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:29.549363 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:31.557550 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:31.557519 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-smf2p" May 28 17:19:32.689533 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:32.689500 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h"] May 28 17:19:32.689901 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:32.689719 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" podUID="110e42ac-b57a-4ebc-aaec-24f6d29ec5ba" containerName="manager" containerID="cri-o://0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866" gracePeriod=10 May 28 17:19:32.932525 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:32.932502 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:32.976227 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:32.976159 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l"] May 28 17:19:32.976760 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:32.976741 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110e42ac-b57a-4ebc-aaec-24f6d29ec5ba" containerName="manager" May 28 17:19:32.976802 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:32.976766 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="110e42ac-b57a-4ebc-aaec-24f6d29ec5ba" containerName="manager" May 28 17:19:32.976888 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:32.976877 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="110e42ac-b57a-4ebc-aaec-24f6d29ec5ba" containerName="manager" May 28 17:19:32.980503 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:32.980480 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.000049 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.000016 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l"] May 28 17:19:33.068208 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.068180 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nz5j\" (UniqueName: \"kubernetes.io/projected/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-kube-api-access-6nz5j\") pod \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\" (UID: \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\") " May 28 17:19:33.068366 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.068275 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-extensions-socket-volume\") pod \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\" (UID: \"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba\") " May 28 17:19:33.068418 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.068403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/610d46ea-90c8-45cb-b7de-be7e6579f171-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-8767l\" (UID: \"610d46ea-90c8-45cb-b7de-be7e6579f171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.068484 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.068436 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnlcw\" (UniqueName: \"kubernetes.io/projected/610d46ea-90c8-45cb-b7de-be7e6579f171-kube-api-access-vnlcw\") pod \"kuadrant-operator-controller-manager-55c7f4c975-8767l\" (UID: \"610d46ea-90c8-45cb-b7de-be7e6579f171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.068665 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.068642 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "110e42ac-b57a-4ebc-aaec-24f6d29ec5ba" (UID: "110e42ac-b57a-4ebc-aaec-24f6d29ec5ba"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 17:19:33.070278 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.070249 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-kube-api-access-6nz5j" (OuterVolumeSpecName: "kube-api-access-6nz5j") pod "110e42ac-b57a-4ebc-aaec-24f6d29ec5ba" (UID: "110e42ac-b57a-4ebc-aaec-24f6d29ec5ba"). InnerVolumeSpecName "kube-api-access-6nz5j". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 17:19:33.169748 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.169690 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/610d46ea-90c8-45cb-b7de-be7e6579f171-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-8767l\" (UID: \"610d46ea-90c8-45cb-b7de-be7e6579f171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.169748 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.169753 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vnlcw\" (UniqueName: \"kubernetes.io/projected/610d46ea-90c8-45cb-b7de-be7e6579f171-kube-api-access-vnlcw\") pod \"kuadrant-operator-controller-manager-55c7f4c975-8767l\" (UID: \"610d46ea-90c8-45cb-b7de-be7e6579f171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.170012 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.169786 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-extensions-socket-volume\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:19:33.170012 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.169796 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nz5j\" (UniqueName: \"kubernetes.io/projected/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba-kube-api-access-6nz5j\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:19:33.170142 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.170118 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/610d46ea-90c8-45cb-b7de-be7e6579f171-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-8767l\" (UID: \"610d46ea-90c8-45cb-b7de-be7e6579f171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.179142 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.179119 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnlcw\" (UniqueName: \"kubernetes.io/projected/610d46ea-90c8-45cb-b7de-be7e6579f171-kube-api-access-vnlcw\") pod \"kuadrant-operator-controller-manager-55c7f4c975-8767l\" (UID: \"610d46ea-90c8-45cb-b7de-be7e6579f171\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.291388 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.291346 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.430084 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.429993 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l"] May 28 17:19:33.432324 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:19:33.432299 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610d46ea_90c8_45cb_b7de_be7e6579f171.slice/crio-5371e2c2cc44ff35f6ef2373de288f4d7b8094d622d8d22652d4b72ee1fdd2cd WatchSource:0}: Error finding container 5371e2c2cc44ff35f6ef2373de288f4d7b8094d622d8d22652d4b72ee1fdd2cd: Status 404 returned error can't find the container with id 5371e2c2cc44ff35f6ef2373de288f4d7b8094d622d8d22652d4b72ee1fdd2cd May 28 17:19:33.600477 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.600384 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" event={"ID":"610d46ea-90c8-45cb-b7de-be7e6579f171","Type":"ContainerStarted","Data":"d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81"} May 28 17:19:33.600477 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.600425 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" event={"ID":"610d46ea-90c8-45cb-b7de-be7e6579f171","Type":"ContainerStarted","Data":"5371e2c2cc44ff35f6ef2373de288f4d7b8094d622d8d22652d4b72ee1fdd2cd"} May 28 17:19:33.600758 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.600525 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:19:33.601568 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.601540 2566 generic.go:358] "Generic (PLEG): container finished" podID="110e42ac-b57a-4ebc-aaec-24f6d29ec5ba" containerID="0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866" exitCode=0 May 28 17:19:33.601568 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.601564 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" event={"ID":"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba","Type":"ContainerDied","Data":"0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866"} May 28 17:19:33.601729 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.601591 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" May 28 17:19:33.601729 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.601603 2566 scope.go:117] "RemoveContainer" containerID="0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866" May 28 17:19:33.601729 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.601594 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h" event={"ID":"110e42ac-b57a-4ebc-aaec-24f6d29ec5ba","Type":"ContainerDied","Data":"d2ef081398841f4db31e596726f4015ab971f85bda7735baa3820772f6b8501e"} May 28 17:19:33.610088 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.610071 2566 scope.go:117] "RemoveContainer" containerID="0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866" May 28 17:19:33.610356 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:19:33.610338 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866\": container with ID starting with 0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866 not found: ID does not exist" containerID="0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866" May 28 17:19:33.610423 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.610363 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866"} err="failed to get container status \"0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866\": rpc error: code = NotFound desc = could not find container \"0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866\": container with ID starting with 0a371536b89713139e2bb799b653b7e0e04900cdd0bf3296d5f84a023eb47866 not found: ID does not exist" May 28 17:19:33.620922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.620885 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" podStartSLOduration=1.6208732270000001 podStartE2EDuration="1.620873227s" podCreationTimestamp="2026-05-28 17:19:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:19:33.619046915 +0000 UTC m=+583.531439572" watchObservedRunningTime="2026-05-28 17:19:33.620873227 +0000 UTC m=+583.533265880" May 28 17:19:33.640070 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.640036 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h"] May 28 17:19:33.645678 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:33.645649 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-9hw7h"] May 28 17:19:34.642021 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:34.641995 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="110e42ac-b57a-4ebc-aaec-24f6d29ec5ba" path="/var/lib/kubelet/pods/110e42ac-b57a-4ebc-aaec-24f6d29ec5ba/volumes" May 28 17:19:44.608901 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:19:44.608870 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:21:56.021470 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.021390 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr"] May 28 17:21:56.032706 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.032674 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.035281 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.035247 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr"] May 28 17:21:56.035499 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.035284 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 28 17:21:56.035681 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.035317 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 28 17:21:56.036268 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.036244 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" May 28 17:21:56.036398 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.036249 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xgqlk\"" May 28 17:21:56.127606 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.127569 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q"] May 28 17:21:56.130047 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.130023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.130118 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.130068 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.130118 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.130097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8ckn\" (UniqueName: \"kubernetes.io/projected/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-kube-api-access-r8ckn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.130193 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.130164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.130193 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.130184 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.130256 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.130222 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.131233 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.131218 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.133480 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.133456 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" May 28 17:21:56.142244 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.142216 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q"] May 28 17:21:56.231279 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231249 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r8ckn\" (UniqueName: \"kubernetes.io/projected/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-kube-api-access-r8ckn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.231440 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.231440 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.231528 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231514 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.231566 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231540 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.231614 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4f2q\" (UniqueName: \"kubernetes.io/projected/c334debf-ddfc-471a-94e8-b10ced595d12-kube-api-access-x4f2q\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.231654 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231612 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c334debf-ddfc-471a-94e8-b10ced595d12-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.231694 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231657 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.231733 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231696 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.231733 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231720 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.231808 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231761 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.231846 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231803 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.231846 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231812 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.231912 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.231860 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.232085 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.232068 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.233909 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.233878 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.234064 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.234009 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.239312 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.239290 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8ckn\" (UniqueName: \"kubernetes.io/projected/daf691ee-bfb9-4e20-85cb-9f26c1a6c81a-kube-api-access-r8ckn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr\" (UID: \"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.333157 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333074 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x4f2q\" (UniqueName: \"kubernetes.io/projected/c334debf-ddfc-471a-94e8-b10ced595d12-kube-api-access-x4f2q\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.333157 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333123 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c334debf-ddfc-471a-94e8-b10ced595d12-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.333419 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333177 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.333419 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333240 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.333419 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333282 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.333419 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333310 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.333648 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333619 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.333919 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333894 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.334020 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.333969 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.335978 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.335956 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c334debf-ddfc-471a-94e8-b10ced595d12-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.336187 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.336171 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c334debf-ddfc-471a-94e8-b10ced595d12-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.345098 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.345074 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:21:56.347880 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.347858 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4f2q\" (UniqueName: \"kubernetes.io/projected/c334debf-ddfc-471a-94e8-b10ced595d12-kube-api-access-x4f2q\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q\" (UID: \"c334debf-ddfc-471a-94e8-b10ced595d12\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.443491 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.443456 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:21:56.478578 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.478550 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr"] May 28 17:21:56.480435 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:21:56.480398 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf691ee_bfb9_4e20_85cb_9f26c1a6c81a.slice/crio-c282e2343b2f06b51045a8e574b410eecfa2cdd47c0c2f7aa64fbb5be4adf79e WatchSource:0}: Error finding container c282e2343b2f06b51045a8e574b410eecfa2cdd47c0c2f7aa64fbb5be4adf79e: Status 404 returned error can't find the container with id c282e2343b2f06b51045a8e574b410eecfa2cdd47c0c2f7aa64fbb5be4adf79e May 28 17:21:56.580680 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:56.580657 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q"] May 28 17:21:56.582972 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:21:56.582934 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc334debf_ddfc_471a_94e8_b10ced595d12.slice/crio-56444aca02e0dd92e373c3c2fedc2e0f1ca45c613575e1b9e22e83fb3cc84d8c WatchSource:0}: Error finding container 56444aca02e0dd92e373c3c2fedc2e0f1ca45c613575e1b9e22e83fb3cc84d8c: Status 404 returned error can't find the container with id 56444aca02e0dd92e373c3c2fedc2e0f1ca45c613575e1b9e22e83fb3cc84d8c May 28 17:21:57.095196 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:57.095142 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" event={"ID":"c334debf-ddfc-471a-94e8-b10ced595d12","Type":"ContainerStarted","Data":"56444aca02e0dd92e373c3c2fedc2e0f1ca45c613575e1b9e22e83fb3cc84d8c"} May 28 17:21:57.096444 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:21:57.096384 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" event={"ID":"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a","Type":"ContainerStarted","Data":"c282e2343b2f06b51045a8e574b410eecfa2cdd47c0c2f7aa64fbb5be4adf79e"} May 28 17:22:02.128631 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:02.128535 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" event={"ID":"c334debf-ddfc-471a-94e8-b10ced595d12","Type":"ContainerStarted","Data":"de592a2b2e7343faf83d21374f863f222bd38b70b84f2ab36dbd3a4e09ee8e8c"} May 28 17:22:02.130163 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:02.130131 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" event={"ID":"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a","Type":"ContainerStarted","Data":"2c68efc7ad2c451cf95d9e577252cb09f9160ad3c093a675d41fce87fe0e089b"} May 28 17:22:08.153824 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:08.153793 2566 generic.go:358] "Generic (PLEG): container finished" podID="daf691ee-bfb9-4e20-85cb-9f26c1a6c81a" containerID="2c68efc7ad2c451cf95d9e577252cb09f9160ad3c093a675d41fce87fe0e089b" exitCode=0 May 28 17:22:08.154310 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:08.153871 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" event={"ID":"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a","Type":"ContainerDied","Data":"2c68efc7ad2c451cf95d9e577252cb09f9160ad3c093a675d41fce87fe0e089b"} May 28 17:22:08.154612 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:08.154584 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 17:22:08.155308 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:08.155174 2566 generic.go:358] "Generic (PLEG): container finished" podID="c334debf-ddfc-471a-94e8-b10ced595d12" containerID="de592a2b2e7343faf83d21374f863f222bd38b70b84f2ab36dbd3a4e09ee8e8c" exitCode=0 May 28 17:22:08.155308 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:08.155203 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" event={"ID":"c334debf-ddfc-471a-94e8-b10ced595d12","Type":"ContainerDied","Data":"de592a2b2e7343faf83d21374f863f222bd38b70b84f2ab36dbd3a4e09ee8e8c"} May 28 17:22:10.165795 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:10.165752 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" event={"ID":"c334debf-ddfc-471a-94e8-b10ced595d12","Type":"ContainerStarted","Data":"7b0f92be433a76e07185e31260bb774d8d1fccb1e4a08989c27e00d62998bc89"} May 28 17:22:10.166251 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:10.166007 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:22:10.167260 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:10.167237 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" event={"ID":"daf691ee-bfb9-4e20-85cb-9f26c1a6c81a","Type":"ContainerStarted","Data":"181134db9fe1f6f121ce816e1146a2b712be5cc24c76507f8d8fc14e8473e425"} May 28 17:22:10.167420 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:10.167405 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:22:10.185617 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:10.185569 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" podStartSLOduration=1.328134285 podStartE2EDuration="14.185556737s" podCreationTimestamp="2026-05-28 17:21:56 +0000 UTC" firstStartedPulling="2026-05-28 17:21:56.584854923 +0000 UTC m=+726.497247553" lastFinishedPulling="2026-05-28 17:22:09.442277371 +0000 UTC m=+739.354670005" observedRunningTime="2026-05-28 17:22:10.184687058 +0000 UTC m=+740.097079726" watchObservedRunningTime="2026-05-28 17:22:10.185556737 +0000 UTC m=+740.097949416" May 28 17:22:10.203094 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:10.203044 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" podStartSLOduration=1.249253063 podStartE2EDuration="14.203028058s" podCreationTimestamp="2026-05-28 17:21:56 +0000 UTC" firstStartedPulling="2026-05-28 17:21:56.48238938 +0000 UTC m=+726.394782011" lastFinishedPulling="2026-05-28 17:22:09.436164356 +0000 UTC m=+739.348557006" observedRunningTime="2026-05-28 17:22:10.201697083 +0000 UTC m=+740.114089736" watchObservedRunningTime="2026-05-28 17:22:10.203028058 +0000 UTC m=+740.115420712" May 28 17:22:21.184425 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:21.184391 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q" May 28 17:22:21.185391 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:21.185368 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr" May 28 17:22:39.524429 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.524391 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf"] May 28 17:22:39.527134 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.527117 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.529474 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.529452 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" May 28 17:22:39.539103 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.539080 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf"] May 28 17:22:39.636859 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.636827 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.637039 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.636916 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.637039 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.636962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.637118 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.637060 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5pcq\" (UniqueName: \"kubernetes.io/projected/1de2d4fe-dcbe-4d66-b548-4b9295474607-kube-api-access-h5pcq\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.637118 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.637106 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.637199 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.637186 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1de2d4fe-dcbe-4d66-b548-4b9295474607-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738318 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738289 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1de2d4fe-dcbe-4d66-b548-4b9295474607-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738318 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738321 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738572 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738344 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738572 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738453 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738572 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738546 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h5pcq\" (UniqueName: \"kubernetes.io/projected/1de2d4fe-dcbe-4d66-b548-4b9295474607-kube-api-access-h5pcq\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738723 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738600 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738723 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738672 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738829 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738718 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.738971 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.738934 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.740553 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.740530 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1de2d4fe-dcbe-4d66-b548-4b9295474607-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.740805 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.740787 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1de2d4fe-dcbe-4d66-b548-4b9295474607-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.747826 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.747801 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5pcq\" (UniqueName: \"kubernetes.io/projected/1de2d4fe-dcbe-4d66-b548-4b9295474607-kube-api-access-h5pcq\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf\" (UID: \"1de2d4fe-dcbe-4d66-b548-4b9295474607\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.837881 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.837795 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:39.975037 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:39.975011 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf"] May 28 17:22:39.977233 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:22:39.977200 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de2d4fe_dcbe_4d66_b548_4b9295474607.slice/crio-cde509527a572822f512f5b34a5fa6da4c684acd0e212f645260960da4b2fe22 WatchSource:0}: Error finding container cde509527a572822f512f5b34a5fa6da4c684acd0e212f645260960da4b2fe22: Status 404 returned error can't find the container with id cde509527a572822f512f5b34a5fa6da4c684acd0e212f645260960da4b2fe22 May 28 17:22:40.272962 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:40.272921 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" event={"ID":"1de2d4fe-dcbe-4d66-b548-4b9295474607","Type":"ContainerStarted","Data":"5fdbf5e0c17ea0bb0ac6df600229f5bd79cd0e3b504c7d3c01966546f1e195b8"} May 28 17:22:40.273147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:40.272983 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" event={"ID":"1de2d4fe-dcbe-4d66-b548-4b9295474607","Type":"ContainerStarted","Data":"cde509527a572822f512f5b34a5fa6da4c684acd0e212f645260960da4b2fe22"} May 28 17:22:46.296464 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:46.296429 2566 generic.go:358] "Generic (PLEG): container finished" podID="1de2d4fe-dcbe-4d66-b548-4b9295474607" containerID="5fdbf5e0c17ea0bb0ac6df600229f5bd79cd0e3b504c7d3c01966546f1e195b8" exitCode=0 May 28 17:22:46.296834 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:46.296503 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" event={"ID":"1de2d4fe-dcbe-4d66-b548-4b9295474607","Type":"ContainerDied","Data":"5fdbf5e0c17ea0bb0ac6df600229f5bd79cd0e3b504c7d3c01966546f1e195b8"} May 28 17:22:47.302158 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:47.302126 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" event={"ID":"1de2d4fe-dcbe-4d66-b548-4b9295474607","Type":"ContainerStarted","Data":"61be4a97fcbcb2f8e13404bb68992ea1deb3808e9a43f3a96e04c3f627bd44bd"} May 28 17:22:47.302602 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:47.302347 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:22:47.321378 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:47.321330 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" podStartSLOduration=7.986424775 podStartE2EDuration="8.321317018s" podCreationTimestamp="2026-05-28 17:22:39 +0000 UTC" firstStartedPulling="2026-05-28 17:22:46.297137421 +0000 UTC m=+776.209530052" lastFinishedPulling="2026-05-28 17:22:46.632029656 +0000 UTC m=+776.544422295" observedRunningTime="2026-05-28 17:22:47.318964117 +0000 UTC m=+777.231356765" watchObservedRunningTime="2026-05-28 17:22:47.321317018 +0000 UTC m=+777.233709670" May 28 17:22:58.319123 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:22:58.319089 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf" May 28 17:34:23.148098 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.148062 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l"] May 28 17:34:23.150607 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.148299 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" podUID="610d46ea-90c8-45cb-b7de-be7e6579f171" containerName="manager" containerID="cri-o://d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81" gracePeriod=10 May 28 17:34:23.587456 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.587434 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:34:23.659091 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.659065 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/610d46ea-90c8-45cb-b7de-be7e6579f171-extensions-socket-volume\") pod \"610d46ea-90c8-45cb-b7de-be7e6579f171\" (UID: \"610d46ea-90c8-45cb-b7de-be7e6579f171\") " May 28 17:34:23.659091 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.659095 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnlcw\" (UniqueName: \"kubernetes.io/projected/610d46ea-90c8-45cb-b7de-be7e6579f171-kube-api-access-vnlcw\") pod \"610d46ea-90c8-45cb-b7de-be7e6579f171\" (UID: \"610d46ea-90c8-45cb-b7de-be7e6579f171\") " May 28 17:34:23.659765 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.659491 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/610d46ea-90c8-45cb-b7de-be7e6579f171-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "610d46ea-90c8-45cb-b7de-be7e6579f171" (UID: "610d46ea-90c8-45cb-b7de-be7e6579f171"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 17:34:23.661315 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.661288 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/610d46ea-90c8-45cb-b7de-be7e6579f171-kube-api-access-vnlcw" (OuterVolumeSpecName: "kube-api-access-vnlcw") pod "610d46ea-90c8-45cb-b7de-be7e6579f171" (UID: "610d46ea-90c8-45cb-b7de-be7e6579f171"). InnerVolumeSpecName "kube-api-access-vnlcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 17:34:23.755548 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.755519 2566 generic.go:358] "Generic (PLEG): container finished" podID="610d46ea-90c8-45cb-b7de-be7e6579f171" containerID="d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81" exitCode=0 May 28 17:34:23.755710 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.755587 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" May 28 17:34:23.755710 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.755606 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" event={"ID":"610d46ea-90c8-45cb-b7de-be7e6579f171","Type":"ContainerDied","Data":"d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81"} May 28 17:34:23.755710 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.755654 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l" event={"ID":"610d46ea-90c8-45cb-b7de-be7e6579f171","Type":"ContainerDied","Data":"5371e2c2cc44ff35f6ef2373de288f4d7b8094d622d8d22652d4b72ee1fdd2cd"} May 28 17:34:23.755710 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.755673 2566 scope.go:117] "RemoveContainer" containerID="d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81" May 28 17:34:23.760169 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.760138 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/610d46ea-90c8-45cb-b7de-be7e6579f171-extensions-socket-volume\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:34:23.760169 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.760167 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnlcw\" (UniqueName: \"kubernetes.io/projected/610d46ea-90c8-45cb-b7de-be7e6579f171-kube-api-access-vnlcw\") on node \"ip-10-0-130-46.ec2.internal\" DevicePath \"\"" May 28 17:34:23.764870 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.764848 2566 scope.go:117] "RemoveContainer" containerID="d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81" May 28 17:34:23.765146 ip-10-0-130-46 kubenswrapper[2566]: E0528 17:34:23.765127 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81\": container with ID starting with d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81 not found: ID does not exist" containerID="d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81" May 28 17:34:23.765202 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.765153 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81"} err="failed to get container status \"d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81\": rpc error: code = NotFound desc = could not find container \"d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81\": container with ID starting with d29bab8245b603959a65aa6b6c8e22474673cef2ab641c6d93429717609ffe81 not found: ID does not exist" May 28 17:34:23.777657 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.777631 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l"] May 28 17:34:23.783129 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:23.783109 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-8767l"] May 28 17:34:24.642053 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:34:24.642024 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="610d46ea-90c8-45cb-b7de-be7e6579f171" path="/var/lib/kubelet/pods/610d46ea-90c8-45cb-b7de-be7e6579f171/volumes" May 28 17:35:29.270120 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.270083 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2"] May 28 17:35:29.270570 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.270461 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="610d46ea-90c8-45cb-b7de-be7e6579f171" containerName="manager" May 28 17:35:29.270570 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.270471 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="610d46ea-90c8-45cb-b7de-be7e6579f171" containerName="manager" May 28 17:35:29.270570 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.270538 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="610d46ea-90c8-45cb-b7de-be7e6579f171" containerName="manager" May 28 17:35:29.273735 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.273719 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:29.277461 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.277439 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9r5b8\"" May 28 17:35:29.285147 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.285120 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2"] May 28 17:35:29.374867 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.374829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sjpz\" (UniqueName: \"kubernetes.io/projected/4d48e413-16e8-4c2b-a5e5-62cde52330ab-kube-api-access-4sjpz\") pod \"kuadrant-operator-controller-manager-55c7f4c975-tx8s2\" (UID: \"4d48e413-16e8-4c2b-a5e5-62cde52330ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:29.375077 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.374912 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d48e413-16e8-4c2b-a5e5-62cde52330ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-tx8s2\" (UID: \"4d48e413-16e8-4c2b-a5e5-62cde52330ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:29.476471 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.476437 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4sjpz\" (UniqueName: \"kubernetes.io/projected/4d48e413-16e8-4c2b-a5e5-62cde52330ab-kube-api-access-4sjpz\") pod \"kuadrant-operator-controller-manager-55c7f4c975-tx8s2\" (UID: \"4d48e413-16e8-4c2b-a5e5-62cde52330ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:29.476645 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.476509 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d48e413-16e8-4c2b-a5e5-62cde52330ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-tx8s2\" (UID: \"4d48e413-16e8-4c2b-a5e5-62cde52330ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:29.476831 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.476815 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d48e413-16e8-4c2b-a5e5-62cde52330ab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-tx8s2\" (UID: \"4d48e413-16e8-4c2b-a5e5-62cde52330ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:29.484482 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.484459 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sjpz\" (UniqueName: \"kubernetes.io/projected/4d48e413-16e8-4c2b-a5e5-62cde52330ab-kube-api-access-4sjpz\") pod \"kuadrant-operator-controller-manager-55c7f4c975-tx8s2\" (UID: \"4d48e413-16e8-4c2b-a5e5-62cde52330ab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:29.584626 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.584539 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:29.721561 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.721538 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2"] May 28 17:35:29.723797 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:35:29.723770 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d48e413_16e8_4c2b_a5e5_62cde52330ab.slice/crio-c824cd834b63461d7f2eaaed7df407235ad2a18ef312c10cd1f9d91ad02dc5a2 WatchSource:0}: Error finding container c824cd834b63461d7f2eaaed7df407235ad2a18ef312c10cd1f9d91ad02dc5a2: Status 404 returned error can't find the container with id c824cd834b63461d7f2eaaed7df407235ad2a18ef312c10cd1f9d91ad02dc5a2 May 28 17:35:29.726009 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.725990 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 17:35:29.988182 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.988093 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" event={"ID":"4d48e413-16e8-4c2b-a5e5-62cde52330ab","Type":"ContainerStarted","Data":"b2ac3970a3c1115e758b47f60138520321ec3c7088241b20d82e7d9d1c7061b5"} May 28 17:35:29.988182 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.988130 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" event={"ID":"4d48e413-16e8-4c2b-a5e5-62cde52330ab","Type":"ContainerStarted","Data":"c824cd834b63461d7f2eaaed7df407235ad2a18ef312c10cd1f9d91ad02dc5a2"} May 28 17:35:29.988182 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:29.988156 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:35:30.011281 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:30.011229 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" podStartSLOduration=1.011213542 podStartE2EDuration="1.011213542s" podCreationTimestamp="2026-05-28 17:35:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:35:30.009847106 +0000 UTC m=+1539.922239755" watchObservedRunningTime="2026-05-28 17:35:30.011213542 +0000 UTC m=+1539.923606195" May 28 17:35:40.994604 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:35:40.994570 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-tx8s2" May 28 17:45:13.450333 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:13.450303 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64688695c6-c6bkw_20dd6b96-7793-4fa7-94e8-08c44febd7fb/manager/0.log" May 28 17:45:15.050044 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:15.050016 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-s6qmg_a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3/manager/0.log" May 28 17:45:15.377415 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:15.377332 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ldp46_09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0/registry-server/0.log" May 28 17:45:15.501223 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:15.501195 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-tx8s2_4d48e413-16e8-4c2b-a5e5-62cde52330ab/manager/0.log" May 28 17:45:15.724093 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:15.723998 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-smf2p_4cbf0533-9be5-45ba-a62c-39093f09a8a4/manager/0.log" May 28 17:45:16.057697 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:16.057673 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-595b7776f86zh8l_61354755-75b9-468a-ac38-042903b1b177/istio-proxy/0.log" May 28 17:45:16.725272 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:16.725245 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d5f85c4d6-gs4zx_ca21983b-3973-4275-86f0-65d6fb0410f1/router/0.log" May 28 17:45:17.065021 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:17.064991 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf_1de2d4fe-dcbe-4d66-b548-4b9295474607/main/0.log" May 28 17:45:17.071745 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:17.071722 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-7cmtf_1de2d4fe-dcbe-4d66-b548-4b9295474607/storage-initializer/0.log" May 28 17:45:17.420705 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:17.420583 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr_daf691ee-bfb9-4e20-85cb-9f26c1a6c81a/storage-initializer/0.log" May 28 17:45:17.427744 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:17.427722 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccdxfkr_daf691ee-bfb9-4e20-85cb-9f26c1a6c81a/main/0.log" May 28 17:45:17.540843 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:17.540811 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q_c334debf-ddfc-471a-94e8-b10ced595d12/storage-initializer/0.log" May 28 17:45:17.550551 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:17.550527 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-sr54q_c334debf-ddfc-471a-94e8-b10ced595d12/main/0.log" May 28 17:45:24.436126 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:24.436094 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j7hsv_5a97db59-58a0-4343-9022-a4da33d6f37e/global-pull-secret-syncer/0.log" May 28 17:45:24.547785 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:24.547754 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g99r4_d8b08027-e6a6-4103-85bd-8ac9b940b103/konnectivity-agent/0.log" May 28 17:45:24.598256 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:24.598226 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-46.ec2.internal_5866c7b02ae35c0d688802a9a7d31ed6/haproxy/0.log" May 28 17:45:29.052633 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:29.052599 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-s6qmg_a2c8cb9a-36d2-4b92-81e8-6f6803ed9cb3/manager/0.log" May 28 17:45:29.131799 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:29.131765 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ldp46_09ac591a-bf75-4b7c-ba74-54ae1f0aa6b0/registry-server/0.log" May 28 17:45:29.194925 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:29.194857 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-tx8s2_4d48e413-16e8-4c2b-a5e5-62cde52330ab/manager/0.log" May 28 17:45:29.272161 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:29.272127 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-smf2p_4cbf0533-9be5-45ba-a62c-39093f09a8a4/manager/0.log" May 28 17:45:30.640406 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:30.640378 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f64ab61a-2865-46ac-822e-8a5d31318958/alertmanager/0.log" May 28 17:45:30.663170 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:30.663150 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f64ab61a-2865-46ac-822e-8a5d31318958/config-reloader/0.log" May 28 17:45:30.690813 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:30.690790 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f64ab61a-2865-46ac-822e-8a5d31318958/kube-rbac-proxy-web/0.log" May 28 17:45:30.711834 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:30.711814 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f64ab61a-2865-46ac-822e-8a5d31318958/kube-rbac-proxy/0.log" May 28 17:45:30.730536 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:30.730512 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f64ab61a-2865-46ac-822e-8a5d31318958/kube-rbac-proxy-metric/0.log" May 28 17:45:30.753885 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:30.753858 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f64ab61a-2865-46ac-822e-8a5d31318958/prom-label-proxy/0.log" May 28 17:45:30.773961 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:30.773928 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f64ab61a-2865-46ac-822e-8a5d31318958/init-config-reloader/0.log" May 28 17:45:31.032916 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.032892 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56569f4c5-htn9m_e0e3d7a0-2fb0-4575-9d09-4a6642dafa4b/metrics-server/0.log" May 28 17:45:31.054551 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.054524 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-std88_70f58936-9848-4a04-b2a8-42e4333c7fec/monitoring-plugin/0.log" May 28 17:45:31.209922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.209895 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zwtlz_37d65077-20e2-4f30-9f1f-42a66f11c959/node-exporter/0.log" May 28 17:45:31.228467 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.228447 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zwtlz_37d65077-20e2-4f30-9f1f-42a66f11c959/kube-rbac-proxy/0.log" May 28 17:45:31.250936 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.250914 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zwtlz_37d65077-20e2-4f30-9f1f-42a66f11c959/init-textfile/0.log" May 28 17:45:31.361423 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.361342 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_32dc81a7-55bb-4586-84f7-c895928c6f26/prometheus/0.log" May 28 17:45:31.379796 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.379774 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_32dc81a7-55bb-4586-84f7-c895928c6f26/config-reloader/0.log" May 28 17:45:31.397960 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.397926 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_32dc81a7-55bb-4586-84f7-c895928c6f26/thanos-sidecar/0.log" May 28 17:45:31.418521 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.418494 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_32dc81a7-55bb-4586-84f7-c895928c6f26/kube-rbac-proxy-web/0.log" May 28 17:45:31.435619 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.435587 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_32dc81a7-55bb-4586-84f7-c895928c6f26/kube-rbac-proxy/0.log" May 28 17:45:31.457834 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.457807 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_32dc81a7-55bb-4586-84f7-c895928c6f26/kube-rbac-proxy-thanos/0.log" May 28 17:45:31.477032 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.477006 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_32dc81a7-55bb-4586-84f7-c895928c6f26/init-config-reloader/0.log" May 28 17:45:31.544990 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.544968 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-f69ld_c4b5ab3a-ef21-4f99-8d9b-6215df555111/prometheus-operator-admission-webhook/0.log" May 28 17:45:31.649162 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.649070 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9cd59bcd-z848k_2682f774-cded-415f-a4a3-bafb74e0b6b3/thanos-query/0.log" May 28 17:45:31.669649 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.669619 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9cd59bcd-z848k_2682f774-cded-415f-a4a3-bafb74e0b6b3/kube-rbac-proxy-web/0.log" May 28 17:45:31.690536 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.690511 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9cd59bcd-z848k_2682f774-cded-415f-a4a3-bafb74e0b6b3/kube-rbac-proxy/0.log" May 28 17:45:31.709894 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.709871 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9cd59bcd-z848k_2682f774-cded-415f-a4a3-bafb74e0b6b3/prom-label-proxy/0.log" May 28 17:45:31.728261 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.728239 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9cd59bcd-z848k_2682f774-cded-415f-a4a3-bafb74e0b6b3/kube-rbac-proxy-rules/0.log" May 28 17:45:31.745611 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:31.745580 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9cd59bcd-z848k_2682f774-cded-415f-a4a3-bafb74e0b6b3/kube-rbac-proxy-metrics/0.log" May 28 17:45:32.944118 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.944040 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk"] May 28 17:45:32.946615 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.946595 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:32.948711 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.948695 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4njhz\"/\"openshift-service-ca.crt\"" May 28 17:45:32.949554 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.949532 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4njhz\"/\"default-dockercfg-4gln2\"" May 28 17:45:32.949554 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.949554 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4njhz\"/\"kube-root-ca.crt\"" May 28 17:45:32.954725 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.954704 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk"] May 28 17:45:32.971924 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.971897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-lib-modules\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:32.972076 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.972005 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-sys\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:32.972076 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.972030 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-podres\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:32.972076 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.972066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-proc\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:32.972305 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:32.972104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r59fp\" (UniqueName: \"kubernetes.io/projected/75124bec-a58d-4dec-934a-5f74a2770ad3-kube-api-access-r59fp\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073042 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073002 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-sys\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073042 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073040 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-podres\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073062 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-proc\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073100 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r59fp\" (UniqueName: \"kubernetes.io/projected/75124bec-a58d-4dec-934a-5f74a2770ad3-kube-api-access-r59fp\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073138 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-sys\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073174 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-lib-modules\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073182 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-podres\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073285 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073194 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-proc\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.073499 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.073332 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75124bec-a58d-4dec-934a-5f74a2770ad3-lib-modules\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.081415 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.081392 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r59fp\" (UniqueName: \"kubernetes.io/projected/75124bec-a58d-4dec-934a-5f74a2770ad3-kube-api-access-r59fp\") pod \"perf-node-gather-daemonset-pknfk\" (UID: \"75124bec-a58d-4dec-934a-5f74a2770ad3\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.256647 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.256621 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:33.387467 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.387438 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk"] May 28 17:45:33.389891 ip-10-0-130-46 kubenswrapper[2566]: W0528 17:45:33.389863 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod75124bec_a58d_4dec_934a_5f74a2770ad3.slice/crio-aab07356e12abd8aa913e94d50c18462576a5941bc039a2fee54056c244bab4a WatchSource:0}: Error finding container aab07356e12abd8aa913e94d50c18462576a5941bc039a2fee54056c244bab4a: Status 404 returned error can't find the container with id aab07356e12abd8aa913e94d50c18462576a5941bc039a2fee54056c244bab4a May 28 17:45:33.391441 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.391424 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 17:45:33.809698 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:33.809671 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-q6g5h_d1f7c3ef-49d9-44ed-b0ed-036939e5ef44/download-server/0.log" May 28 17:45:34.096468 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:34.096383 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" event={"ID":"75124bec-a58d-4dec-934a-5f74a2770ad3","Type":"ContainerStarted","Data":"9cc875c8f95ef1147f5ca52b6615e3c9bbb4768013d3dfdca6c77f61482e1b9f"} May 28 17:45:34.096468 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:34.096419 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" event={"ID":"75124bec-a58d-4dec-934a-5f74a2770ad3","Type":"ContainerStarted","Data":"aab07356e12abd8aa913e94d50c18462576a5941bc039a2fee54056c244bab4a"} May 28 17:45:34.096860 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:34.096499 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:34.111161 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:34.111115 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" podStartSLOduration=2.111103349 podStartE2EDuration="2.111103349s" podCreationTimestamp="2026-05-28 17:45:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:45:34.110105919 +0000 UTC m=+2144.022498582" watchObservedRunningTime="2026-05-28 17:45:34.111103349 +0000 UTC m=+2144.023496049" May 28 17:45:34.290268 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:34.290237 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-spj4p_b0750930-f62e-4151-83ec-ee92469cfa02/volume-data-source-validator/0.log" May 28 17:45:35.125292 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:35.125265 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h68jj_b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c/dns/0.log" May 28 17:45:35.142967 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:35.142924 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h68jj_b5e7bbc9-16f7-4cbb-a3b4-c90b14a9428c/kube-rbac-proxy/0.log" May 28 17:45:35.161930 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:35.161905 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4frtv_37b8dbdd-d75f-47e5-8776-fbb29a0f2104/dns-node-resolver/0.log" May 28 17:45:35.676976 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:35.676932 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-747fdf66db-spdfc_960a36d4-962b-465b-8376-a1c6e1101a44/registry/0.log" May 28 17:45:35.713899 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:35.713870 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-szzhz_a73de68c-3175-4736-bf34-52f955c17308/node-ca/0.log" May 28 17:45:36.510807 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:36.510777 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-595b7776f86zh8l_61354755-75b9-468a-ac38-042903b1b177/istio-proxy/0.log" May 28 17:45:36.832574 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:36.832491 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d5f85c4d6-gs4zx_ca21983b-3973-4275-86f0-65d6fb0410f1/router/0.log" May 28 17:45:37.321471 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:37.321438 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rqq5s_9e5b6585-37d1-4e39-8599-509b085c834a/serve-healthcheck-canary/0.log" May 28 17:45:37.845828 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:37.845797 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qc67w_ed52b9e1-36c2-4a65-9f38-edc2790fc1e0/kube-rbac-proxy/0.log" May 28 17:45:37.863011 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:37.862989 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qc67w_ed52b9e1-36c2-4a65-9f38-edc2790fc1e0/exporter/0.log" May 28 17:45:37.886250 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:37.886223 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qc67w_ed52b9e1-36c2-4a65-9f38-edc2790fc1e0/extractor/0.log" May 28 17:45:39.947267 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:39.947238 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64688695c6-c6bkw_20dd6b96-7793-4fa7-94e8-08c44febd7fb/manager/0.log" May 28 17:45:40.109556 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:40.109531 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-pknfk" May 28 17:45:41.052797 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:41.052767 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7dc4c66bd5-lshzt_ce322ec3-5cb5-4e80-8bc4-14ca438f06ed/manager/0.log" May 28 17:45:45.781670 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:45.781638 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-nm9j9_b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d/kube-storage-version-migrator-operator/1.log" May 28 17:45:45.782515 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:45.782496 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-nm9j9_b9bf3aa0-0375-4ecb-afe6-51a56dd8d31d/kube-storage-version-migrator-operator/0.log" May 28 17:45:46.863329 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:46.863297 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlvsd_e7653292-2abd-4283-b387-3ad935b2be56/kube-multus-additional-cni-plugins/0.log" May 28 17:45:46.881023 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:46.880995 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlvsd_e7653292-2abd-4283-b387-3ad935b2be56/egress-router-binary-copy/0.log" May 28 17:45:46.898228 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:46.898203 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlvsd_e7653292-2abd-4283-b387-3ad935b2be56/cni-plugins/0.log" May 28 17:45:46.915161 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:46.915140 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlvsd_e7653292-2abd-4283-b387-3ad935b2be56/bond-cni-plugin/0.log" May 28 17:45:46.932171 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:46.932154 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlvsd_e7653292-2abd-4283-b387-3ad935b2be56/routeoverride-cni/0.log" May 28 17:45:46.950420 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:46.950396 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlvsd_e7653292-2abd-4283-b387-3ad935b2be56/whereabouts-cni-bincopy/0.log" May 28 17:45:46.975864 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:46.975839 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vlvsd_e7653292-2abd-4283-b387-3ad935b2be56/whereabouts-cni/0.log" May 28 17:45:47.181683 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:47.181615 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dgbcd_fc76295c-2e54-4682-a401-5c84d779932a/kube-multus/0.log" May 28 17:45:47.239922 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:47.239893 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lsl89_e04539f3-03d0-41ac-8c13-0f67edc8e25c/network-metrics-daemon/0.log" May 28 17:45:47.257965 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:47.257934 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lsl89_e04539f3-03d0-41ac-8c13-0f67edc8e25c/kube-rbac-proxy/0.log" May 28 17:45:48.097243 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:48.097217 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dnjtt_1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264/ovn-controller/0.log" May 28 17:45:48.124281 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:48.124254 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dnjtt_1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264/ovn-acl-logging/0.log" May 28 17:45:48.144184 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:48.144153 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dnjtt_1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264/kube-rbac-proxy-node/0.log" May 28 17:45:48.162353 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:48.162329 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dnjtt_1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264/kube-rbac-proxy-ovn-metrics/0.log" May 28 17:45:48.178695 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:48.178672 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dnjtt_1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264/northd/0.log" May 28 17:45:48.196747 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:48.196718 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dnjtt_1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264/nbdb/0.log" May 28 17:45:48.214470 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:48.214451 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dnjtt_1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264/sbdb/0.log" May 28 17:45:48.307816 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:48.307787 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dnjtt_1e7e77b5-ba57-4a6b-bdfa-9c546c6fa264/ovnkube-controller/0.log" May 28 17:45:49.900875 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:49.900852 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-r8pdq_f8bee3dd-7589-414a-b93a-816a92414e88/check-endpoints/0.log" May 28 17:45:49.923762 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:49.923736 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-kxlwn_5d76de2f-eb8b-4811-843f-8cdaa2dca0ce/network-check-target-container/0.log" May 28 17:45:50.982037 ip-10-0-130-46 kubenswrapper[2566]: I0528 17:45:50.982002 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-xnpdv_abe63baf-7676-4084-9037-525c520b3707/iptables-alerter/0.log"