May 29 00:44:38.602299 ip-10-0-141-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 29 00:44:38.602313 ip-10-0-141-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 29 00:44:38.602322 ip-10-0-141-236 systemd[1]: kubelet.service: Failed with result 'resources'. May 29 00:44:38.602599 ip-10-0-141-236 systemd[1]: Failed to start Kubernetes Kubelet. May 29 00:44:48.714040 ip-10-0-141-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 29 00:44:48.714062 ip-10-0-141-236 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a2baa6a3553d4308812896d0845241a4 -- May 29 00:47:21.410073 ip-10-0-141-236 systemd[1]: Starting Kubernetes Kubelet... May 29 00:47:21.914856 ip-10-0-141-236 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 29 00:47:21.914856 ip-10-0-141-236 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 29 00:47:21.914856 ip-10-0-141-236 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 29 00:47:21.914856 ip-10-0-141-236 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 29 00:47:21.917923 ip-10-0-141-236 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 29 00:47:21.919784 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.919689 2551 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 29 00:47:21.923664 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923643 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 00:47:21.923664 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923660 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 00:47:21.923664 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923664 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 00:47:21.923664 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923668 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 00:47:21.923664 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923671 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923673 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923676 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923679 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923681 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923684 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923687 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923690 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923693 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923695 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923698 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923700 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923703 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923705 2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923708 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923711 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923713 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923716 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923725 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 00:47:21.923874 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923728 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923730 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923733 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923735 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923738 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923740 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923742 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923745 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923747 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923762 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923765 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923768 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923771 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923773 2551 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923776 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923779 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923782 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923785 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923787 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923790 2551 feature_gate.go:328] unrecognized feature gate: Example May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923792 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 00:47:21.924323 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923795 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923797 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923800 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923802 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923805 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923808 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923810 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923814 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923816 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923819 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923821 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923824 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923826 2551 feature_gate.go:328] unrecognized feature gate: Example2 May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923829 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923831 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923836 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923839 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923842 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923847 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 00:47:21.924845 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923850 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923853 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923856 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923859 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923862 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923864 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923867 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923870 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923873 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923876 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923878 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923881 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923884 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923887 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923890 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923893 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923897 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923899 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923902 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923905 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 00:47:21.925288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923907 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923910 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.923912 2551 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924286 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924291 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924293 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924296 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924299 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924301 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924304 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924306 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924309 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924312 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924314 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924317 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924319 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924322 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924324 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924327 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924330 2551 feature_gate.go:328] unrecognized feature gate: Example2 May 29 00:47:21.925768 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924333 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924336 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924338 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924341 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924343 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924346 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924349 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924352 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924354 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924357 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924359 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924362 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924364 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924367 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924369 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924372 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924374 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924377 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924379 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924381 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 00:47:21.926259 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924384 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924386 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924389 2551 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924391 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924394 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924396 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924398 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924401 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924403 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924406 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924408 2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924411 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924414 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924416 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924420 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924423 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924432 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924435 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924438 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 00:47:21.926759 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924441 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924443 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924446 2551 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924449 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924451 2551 feature_gate.go:328] unrecognized feature gate: Example May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924455 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924457 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924459 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924462 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924464 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924467 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924469 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924472 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924474 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924477 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924479 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924481 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924485 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924487 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924490 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 00:47:21.927220 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924493 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924496 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924498 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924501 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924503 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924506 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924508 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924511 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924513 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.924516 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924583 2551 flags.go:64] FLAG: --address="0.0.0.0" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924593 2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924599 2551 flags.go:64] FLAG: --anonymous-auth="true" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924604 2551 flags.go:64] FLAG: --application-metrics-count-limit="100" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924616 2551 flags.go:64] FLAG: --authentication-token-webhook="false" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924619 2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924623 2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924627 2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924630 2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924633 2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924637 2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924640 2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 29 00:47:21.927705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924643 2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924645 2551 flags.go:64] FLAG: --cgroup-root="" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924648 2551 flags.go:64] FLAG: --cgroups-per-qos="true" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924651 2551 flags.go:64] FLAG: --client-ca-file="" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924654 2551 flags.go:64] FLAG: --cloud-config="" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924657 2551 flags.go:64] FLAG: --cloud-provider="external" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924660 2551 flags.go:64] FLAG: --cluster-dns="[]" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924664 2551 flags.go:64] FLAG: --cluster-domain="" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924666 2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924669 2551 flags.go:64] FLAG: --config-dir="" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924672 2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924675 2551 flags.go:64] FLAG: --container-log-max-files="5" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924679 2551 flags.go:64] FLAG: --container-log-max-size="10Mi" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924682 2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924685 2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924688 2551 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924691 2551 flags.go:64] FLAG: --contention-profiling="false" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924694 2551 flags.go:64] FLAG: --cpu-cfs-quota="true" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924697 2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924700 2551 flags.go:64] FLAG: --cpu-manager-policy="none" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924703 2551 flags.go:64] FLAG: --cpu-manager-policy-options="" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924709 2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924711 2551 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924714 2551 flags.go:64] FLAG: --enable-debugging-handlers="true" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924717 2551 flags.go:64] FLAG: --enable-load-reader="false" May 29 00:47:21.928262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924720 2551 flags.go:64] FLAG: --enable-server="true" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924723 2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924728 2551 flags.go:64] FLAG: --event-burst="100" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924731 2551 flags.go:64] FLAG: --event-qps="50" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924733 2551 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924736 2551 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924739 2551 flags.go:64] FLAG: --eviction-hard="" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924743 2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924745 2551 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924762 2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924768 2551 flags.go:64] FLAG: --eviction-soft="" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924773 2551 flags.go:64] FLAG: --eviction-soft-grace-period="" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924777 2551 flags.go:64] FLAG: --exit-on-lock-contention="false" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924780 2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924783 2551 flags.go:64] FLAG: --experimental-mounter-path="" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924786 2551 flags.go:64] FLAG: --fail-cgroupv1="false" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924788 2551 flags.go:64] FLAG: --fail-swap-on="true" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924791 2551 flags.go:64] FLAG: --feature-gates="" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924795 2551 flags.go:64] FLAG: --file-check-frequency="20s" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924798 2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924801 2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924804 2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924807 2551 flags.go:64] FLAG: --healthz-port="10248" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924810 2551 flags.go:64] FLAG: --help="false" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924813 2551 flags.go:64] FLAG: --hostname-override="ip-10-0-141-236.ec2.internal" May 29 00:47:21.928888 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924816 2551 flags.go:64] FLAG: --housekeeping-interval="10s" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924819 2551 flags.go:64] FLAG: --http-check-frequency="20s" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924822 2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924826 2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924830 2551 flags.go:64] FLAG: --image-gc-high-threshold="85" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924833 2551 flags.go:64] FLAG: --image-gc-low-threshold="80" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924835 2551 flags.go:64] FLAG: --image-service-endpoint="" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924838 2551 flags.go:64] FLAG: --kernel-memcg-notification="false" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924842 2551 flags.go:64] FLAG: --kube-api-burst="100" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924844 2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924848 2551 flags.go:64] FLAG: --kube-api-qps="50" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924850 2551 flags.go:64] FLAG: --kube-reserved="" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924853 2551 flags.go:64] FLAG: --kube-reserved-cgroup="" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924856 2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924859 2551 flags.go:64] FLAG: --kubelet-cgroups="" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924862 2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924865 2551 flags.go:64] FLAG: --lock-file="" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924868 2551 flags.go:64] FLAG: --log-cadvisor-usage="false" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924870 2551 flags.go:64] FLAG: --log-flush-frequency="5s" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924873 2551 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924878 2551 flags.go:64] FLAG: --log-json-split-stream="false" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924881 2551 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924884 2551 flags.go:64] FLAG: --log-text-split-stream="false" May 29 00:47:21.929495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924886 2551 flags.go:64] FLAG: --logging-format="text" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924889 2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924893 2551 flags.go:64] FLAG: --make-iptables-util-chains="true" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924895 2551 flags.go:64] FLAG: --manifest-url="" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924898 2551 flags.go:64] FLAG: --manifest-url-header="" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924902 2551 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924905 2551 flags.go:64] FLAG: --max-open-files="1000000" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924910 2551 flags.go:64] FLAG: --max-pods="110" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924913 2551 flags.go:64] FLAG: --maximum-dead-containers="-1" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924915 2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924918 2551 flags.go:64] FLAG: --memory-manager-policy="None" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924921 2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924925 2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924928 2551 flags.go:64] FLAG: --node-ip="0.0.0.0" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924931 2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924938 2551 flags.go:64] FLAG: --node-status-max-images="50" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924941 2551 flags.go:64] FLAG: --node-status-update-frequency="10s" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924944 2551 flags.go:64] FLAG: --oom-score-adj="-999" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924954 2551 flags.go:64] FLAG: --pod-cidr="" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924957 2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924963 2551 flags.go:64] FLAG: --pod-manifest-path="" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924966 2551 flags.go:64] FLAG: --pod-max-pids="-1" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924969 2551 flags.go:64] FLAG: --pods-per-core="0" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924972 2551 flags.go:64] FLAG: --port="10250" May 29 00:47:21.930065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924975 2551 flags.go:64] FLAG: --protect-kernel-defaults="false" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924977 2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d79161c84e429514" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924980 2551 flags.go:64] FLAG: --qos-reserved="" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924983 2551 flags.go:64] FLAG: --read-only-port="10255" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924986 2551 flags.go:64] FLAG: --register-node="true" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924989 2551 flags.go:64] FLAG: --register-schedulable="true" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924992 2551 flags.go:64] FLAG: --register-with-taints="" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.924999 2551 flags.go:64] FLAG: --registry-burst="10" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925002 2551 flags.go:64] FLAG: --registry-qps="5" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925005 2551 flags.go:64] FLAG: --reserved-cpus="" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925007 2551 flags.go:64] FLAG: --reserved-memory="" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925011 2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925014 2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925017 2551 flags.go:64] FLAG: --rotate-certificates="false" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925020 2551 flags.go:64] FLAG: --rotate-server-certificates="false" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925022 2551 flags.go:64] FLAG: --runonce="false" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925025 2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925028 2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925031 2551 flags.go:64] FLAG: --seccomp-default="false" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925033 2551 flags.go:64] FLAG: --serialize-image-pulls="true" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925038 2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925041 2551 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925044 2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925047 2551 flags.go:64] FLAG: --storage-driver-password="root" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925050 2551 flags.go:64] FLAG: --storage-driver-secure="false" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925052 2551 flags.go:64] FLAG: --storage-driver-table="stats" May 29 00:47:21.930678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925055 2551 flags.go:64] FLAG: --storage-driver-user="root" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925059 2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925062 2551 flags.go:64] FLAG: --sync-frequency="1m0s" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925064 2551 flags.go:64] FLAG: --system-cgroups="" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925067 2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925072 2551 flags.go:64] FLAG: --system-reserved-cgroup="" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925075 2551 flags.go:64] FLAG: --tls-cert-file="" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925077 2551 flags.go:64] FLAG: --tls-cipher-suites="[]" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925081 2551 flags.go:64] FLAG: --tls-min-version="" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925084 2551 flags.go:64] FLAG: --tls-private-key-file="" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925087 2551 flags.go:64] FLAG: --topology-manager-policy="none" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925090 2551 flags.go:64] FLAG: --topology-manager-policy-options="" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925093 2551 flags.go:64] FLAG: --topology-manager-scope="container" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925095 2551 flags.go:64] FLAG: --v="2" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925099 2551 flags.go:64] FLAG: --version="false" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925103 2551 flags.go:64] FLAG: --vmodule="" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925107 2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925111 2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925202 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925209 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925212 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925215 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925218 2551 feature_gate.go:328] unrecognized feature gate: Example2 May 29 00:47:21.931350 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925222 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925244 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925247 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925253 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925255 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925258 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925261 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925264 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925267 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925270 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925272 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925276 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925278 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925281 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925283 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925287 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925289 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925292 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925295 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925297 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 00:47:21.931905 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925300 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925303 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925305 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925308 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925310 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925313 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925315 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925317 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925321 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925324 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925326 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925329 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925331 2551 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925334 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925336 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925339 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925342 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925344 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925347 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925349 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 00:47:21.932399 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925352 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925355 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925359 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925361 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925364 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925367 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925369 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925372 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925375 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925377 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925379 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925382 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925384 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925387 2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925390 2551 feature_gate.go:328] unrecognized feature gate: Example May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925392 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925394 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925397 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925399 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 00:47:21.932880 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925401 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925405 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925408 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925411 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925413 2551 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925416 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925418 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925421 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925424 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925426 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925429 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925432 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925434 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925437 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925439 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925442 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925444 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925446 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925449 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925452 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 00:47:21.933333 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925454 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.925457 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.925462 2551 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.931688 2551 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.931701 2551 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931766 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931771 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931775 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931778 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931781 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931784 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931787 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931790 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931792 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931795 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 00:47:21.933887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931798 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931800 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931803 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931805 2551 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931808 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931810 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931813 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931815 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931818 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931820 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931824 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931828 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931831 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931834 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931837 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931840 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931843 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931846 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931849 2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 00:47:21.934288 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931852 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931855 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931858 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931860 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931863 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931865 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931869 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931873 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931876 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931878 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931881 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931883 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931886 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931888 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931891 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931893 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931896 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931898 2551 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931900 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931903 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 00:47:21.934735 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931906 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931908 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931912 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931914 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931917 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931920 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931923 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931926 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931928 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931931 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931933 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931936 2551 feature_gate.go:328] unrecognized feature gate: Example May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931938 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931941 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931943 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931946 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931949 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931951 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931954 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931956 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 00:47:21.935232 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931959 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931961 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931964 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931966 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931968 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931971 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931973 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931976 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931978 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931981 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931983 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931986 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931988 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931991 2551 feature_gate.go:328] unrecognized feature gate: Example2 May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931993 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931995 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 00:47:21.935773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.931998 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.932003 2551 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932094 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932099 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932102 2551 feature_gate.go:328] unrecognized feature gate: NewOLM May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932105 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932108 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932111 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932114 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932116 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932119 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932122 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932124 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932127 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932129 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932131 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 29 00:47:21.936163 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932134 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932137 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932139 2551 feature_gate.go:328] unrecognized feature gate: Example2 May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932142 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932144 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932146 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932149 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932152 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932155 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932158 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932160 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932163 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932165 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932168 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932170 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932172 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932175 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932178 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932180 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 29 00:47:21.936555 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932183 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932185 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932188 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932190 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932193 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932196 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932198 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932201 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932203 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932206 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932208 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932210 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932213 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932215 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932218 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932220 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932223 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932225 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932228 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932231 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 29 00:47:21.937015 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932233 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932236 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932239 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932241 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932244 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932246 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932248 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932251 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932253 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932256 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932259 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932262 2551 feature_gate.go:328] unrecognized feature gate: DualReplica May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932265 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932267 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932270 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932273 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932276 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932280 2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932283 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 29 00:47:21.937502 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932286 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932288 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932290 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932293 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932295 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932298 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932300 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932303 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932305 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932308 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932310 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932313 2551 feature_gate.go:328] unrecognized feature gate: Example May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932315 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:21.932317 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.932322 2551 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 29 00:47:21.938062 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.933135 2551 server.go:962] "Client rotation is on, will bootstrap in background" May 29 00:47:21.938428 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.935188 2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 29 00:47:21.938428 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.936218 2551 server.go:1019] "Starting client certificate rotation" May 29 00:47:21.938428 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.936315 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 29 00:47:21.938428 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.936349 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 29 00:47:21.962930 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.962912 2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 29 00:47:21.964921 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.964897 2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 29 00:47:21.975077 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.975059 2551 log.go:25] "Validated CRI v1 runtime API" May 29 00:47:21.982267 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.982242 2551 log.go:25] "Validated CRI v1 image API" May 29 00:47:21.986137 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.986122 2551 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 29 00:47:21.990777 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.990735 2551 fs.go:135] Filesystem UUIDs: map[66731b89-1922-4272-9d20-19675b62a854:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b933334a-39da-4d2a-9af3-38c2ce050be4:/dev/nvme0n1p3] May 29 00:47:21.990843 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.990777 2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 29 00:47:21.995355 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.995248 2551 manager.go:217] Machine: {Timestamp:2026-05-29 00:47:21.994770644 +0000 UTC m=+0.454974083 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199890 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28790fbb987554facf507db4d171b0 SystemUUID:ec28790f-bb98-7554-facf-507db4d171b0 BootID:a2baa6a3-553d-4308-8128-96d0845241a4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:23:1b:0b:fa:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:23:1b:0b:fa:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:eb:a3:4c:0d:ae Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 29 00:47:21.996552 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.996534 2551 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 29 00:47:21.997332 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.997320 2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 29 00:47:21.997413 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.997401 2551 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 29 00:47:21.999647 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.999628 2551 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 29 00:47:21.999797 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.999649 2551 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 29 00:47:21.999841 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.999806 2551 topology_manager.go:138] "Creating topology manager with none policy" May 29 00:47:21.999841 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.999814 2551 container_manager_linux.go:306] "Creating device plugin manager" May 29 00:47:21.999841 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:21.999821 2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 29 00:47:22.000568 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.000558 2551 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 29 00:47:22.002324 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.002313 2551 state_mem.go:36] "Initialized new in-memory state store" May 29 00:47:22.002573 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.002564 2551 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 29 00:47:22.005196 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.005186 2551 kubelet.go:491] "Attempting to sync node with API server" May 29 00:47:22.005230 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.005203 2551 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 29 00:47:22.005230 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.005215 2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 29 00:47:22.005230 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.005224 2551 kubelet.go:397] "Adding apiserver pod source" May 29 00:47:22.005351 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.005232 2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 29 00:47:22.006283 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.006272 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 29 00:47:22.006325 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.006289 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 29 00:47:22.009306 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.009292 2551 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 29 00:47:22.010725 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.010709 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hp8pk" May 29 00:47:22.011562 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.011545 2551 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 29 00:47:22.012396 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012382 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012401 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012410 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012419 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012431 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012439 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012448 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012457 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012466 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 29 00:47:22.012471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012474 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 29 00:47:22.012706 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012497 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 29 00:47:22.012874 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.012862 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 29 00:47:22.014021 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.014009 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 29 00:47:22.014075 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.014024 2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 29 00:47:22.016969 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.016954 2551 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 29 00:47:22.017040 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.016993 2551 server.go:1295] "Started kubelet" May 29 00:47:22.017094 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.017057 2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 29 00:47:22.017396 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.017341 2551 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 29 00:47:22.017465 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.017426 2551 server_v1.go:47] "podresources" method="list" useActivePods=true May 29 00:47:22.017465 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.017423 2551 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 29 00:47:22.017465 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.017421 2551 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 29 00:47:22.017616 ip-10-0-141-236 systemd[1]: Started Kubernetes Kubelet. May 29 00:47:22.017705 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.017612 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hp8pk" May 29 00:47:22.018540 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.018526 2551 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 29 00:47:22.020254 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.020235 2551 server.go:317] "Adding debug handlers to kubelet server" May 29 00:47:22.028337 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.028323 2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 29 00:47:22.028427 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.028341 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 29 00:47:22.028427 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.028399 2551 volume_manager.go:295] "The desired_state_of_world populator starts" May 29 00:47:22.028427 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.028411 2551 volume_manager.go:297] "Starting Kubelet Volume Manager" May 29 00:47:22.028572 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.028502 2551 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 29 00:47:22.028572 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.028562 2551 reconstruct.go:97] "Volume reconstruction finished" May 29 00:47:22.028671 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.028574 2551 reconciler.go:26] "Reconciler: start to sync state" May 29 00:47:22.029106 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.029081 2551 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 29 00:47:22.029192 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.029164 2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 29 00:47:22.029192 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.029178 2551 factory.go:55] Registering systemd factory May 29 00:47:22.029192 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.029186 2551 factory.go:223] Registration of the systemd container factory successfully May 29 00:47:22.029339 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.029248 2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.030712 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.029864 2551 factory.go:153] Registering CRI-O factory May 29 00:47:22.030712 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.029880 2551 factory.go:223] Registration of the crio container factory successfully May 29 00:47:22.030712 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.029896 2551 factory.go:103] Registering Raw factory May 29 00:47:22.030712 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.029908 2551 manager.go:1196] Started watching for new ooms in manager May 29 00:47:22.030712 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.030238 2551 manager.go:319] Starting recovery of all containers May 29 00:47:22.031221 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.031198 2551 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 29 00:47:22.034410 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.034389 2551 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-236.ec2.internal\" not found" node="ip-10-0-141-236.ec2.internal" May 29 00:47:22.035481 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.035292 2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-236.ec2.internal" not found May 29 00:47:22.037629 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.037613 2551 manager.go:324] Recovery completed May 29 00:47:22.038103 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.038074 2551 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 29 00:47:22.042238 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.042225 2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 29 00:47:22.043917 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.043903 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientMemory" May 29 00:47:22.043978 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.043930 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasNoDiskPressure" May 29 00:47:22.043978 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.043941 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientPID" May 29 00:47:22.044352 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.044341 2551 cpu_manager.go:222] "Starting CPU manager" policy="none" May 29 00:47:22.044409 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.044353 2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 29 00:47:22.044409 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.044370 2551 state_mem.go:36] "Initialized new in-memory state store" May 29 00:47:22.046857 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.046846 2551 policy_none.go:49] "None policy: Start" May 29 00:47:22.046907 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.046860 2551 memory_manager.go:186] "Starting memorymanager" policy="None" May 29 00:47:22.046907 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.046870 2551 state_mem.go:35] "Initializing new in-memory state store" May 29 00:47:22.051721 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.051707 2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-236.ec2.internal" not found May 29 00:47:22.088350 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.088337 2551 manager.go:341] "Starting Device Plugin manager" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.088372 2551 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.088393 2551 server.go:85] "Starting device plugin registration server" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.088612 2551 eviction_manager.go:189] "Eviction manager: starting control loop" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.088624 2551 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.088740 2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.088818 2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.088826 2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.089459 2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 29 00:47:22.104317 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.089489 2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.109474 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.109462 2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-236.ec2.internal" not found May 29 00:47:22.136279 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.136262 2551 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 29 00:47:22.136279 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.136281 2551 status_manager.go:230] "Starting to sync pod status with apiserver" May 29 00:47:22.136376 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.136294 2551 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 29 00:47:22.136376 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.136300 2551 kubelet.go:2451] "Starting kubelet main sync loop" May 29 00:47:22.136376 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.136330 2551 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 29 00:47:22.138722 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.138707 2551 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 29 00:47:22.189424 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.189383 2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 29 00:47:22.191703 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.191689 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientMemory" May 29 00:47:22.191788 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.191712 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasNoDiskPressure" May 29 00:47:22.191788 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.191723 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientPID" May 29 00:47:22.191788 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.191741 2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-236.ec2.internal" May 29 00:47:22.200810 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.200795 2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-236.ec2.internal" May 29 00:47:22.200879 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.200814 2551 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-236.ec2.internal\": node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.214843 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.214827 2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.237076 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.237040 2551 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal"] May 29 00:47:22.237167 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.237111 2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 29 00:47:22.237880 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.237867 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientMemory" May 29 00:47:22.237934 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.237890 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasNoDiskPressure" May 29 00:47:22.237934 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.237900 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientPID" May 29 00:47:22.238936 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.238925 2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 29 00:47:22.239070 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.239056 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.239116 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.239083 2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 29 00:47:22.240997 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.240984 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientMemory" May 29 00:47:22.241056 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.241005 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasNoDiskPressure" May 29 00:47:22.241056 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.241019 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientPID" May 29 00:47:22.241056 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.241043 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientMemory" May 29 00:47:22.241156 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.241061 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasNoDiskPressure" May 29 00:47:22.241156 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.241073 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientPID" May 29 00:47:22.242309 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.242296 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" May 29 00:47:22.242354 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.242320 2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 29 00:47:22.243169 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.243155 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientMemory" May 29 00:47:22.243226 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.243183 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasNoDiskPressure" May 29 00:47:22.243226 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.243197 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeHasSufficientPID" May 29 00:47:22.261261 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.261244 2551 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-236.ec2.internal\" not found" node="ip-10-0-141-236.ec2.internal" May 29 00:47:22.264465 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.264447 2551 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-236.ec2.internal\" not found" node="ip-10-0-141-236.ec2.internal" May 29 00:47:22.315233 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.315216 2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.330341 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.330322 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d23c050efd0f50b645c9f513ee415f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal\" (UID: \"c3d23c050efd0f50b645c9f513ee415f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.330428 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.330348 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b9682f407829b9ba775328d34f44bc9c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-236.ec2.internal\" (UID: \"b9682f407829b9ba775328d34f44bc9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" May 29 00:47:22.330428 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.330377 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3d23c050efd0f50b645c9f513ee415f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal\" (UID: \"c3d23c050efd0f50b645c9f513ee415f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.416028 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.416002 2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.431324 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.431306 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b9682f407829b9ba775328d34f44bc9c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-236.ec2.internal\" (UID: \"b9682f407829b9ba775328d34f44bc9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" May 29 00:47:22.431378 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.431330 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3d23c050efd0f50b645c9f513ee415f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal\" (UID: \"c3d23c050efd0f50b645c9f513ee415f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.431378 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.431346 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d23c050efd0f50b645c9f513ee415f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal\" (UID: \"c3d23c050efd0f50b645c9f513ee415f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.431442 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.431391 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d23c050efd0f50b645c9f513ee415f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal\" (UID: \"c3d23c050efd0f50b645c9f513ee415f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.431442 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.431396 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b9682f407829b9ba775328d34f44bc9c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-236.ec2.internal\" (UID: \"b9682f407829b9ba775328d34f44bc9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" May 29 00:47:22.431442 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.431408 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3d23c050efd0f50b645c9f513ee415f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal\" (UID: \"c3d23c050efd0f50b645c9f513ee415f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.516735 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.516683 2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.564129 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.564110 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.567738 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.567724 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" May 29 00:47:22.617631 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.617599 2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.718089 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.718058 2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.818594 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.818520 2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-236.ec2.internal\" not found" May 29 00:47:22.895215 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.895186 2551 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 29 00:47:22.924661 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.924639 2551 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 29 00:47:22.929452 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.929433 2551 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.936326 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.936313 2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 29 00:47:22.936427 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:22.936409 2551 kubelet.go:3342] "Failed creating a mirror pod" err="Post \"https://aea745e4310c548baa67afb9917fe699-c13df39f8deab48a.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.141.236:52386->3.235.213.184:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" May 29 00:47:22.936484 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.936428 2551 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" May 29 00:47:22.936484 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.936461 2551 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 29 00:47:22.936484 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.936472 2551 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 29 00:47:22.936580 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.936466 2551 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 29 00:47:22.936580 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.936486 2551 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 29 00:47:22.952639 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:22.952621 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 29 00:47:23.006227 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.006186 2551 apiserver.go:52] "Watching apiserver" May 29 00:47:23.015556 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.015538 2551 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 29 00:47:23.016500 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.016478 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-jjscg","openshift-multus/network-metrics-daemon-px5qc","openshift-network-diagnostics/network-check-target-6fjlq","openshift-network-operator/iptables-alerter-8x6h4","kube-system/konnectivity-agent-979kv","openshift-dns/node-resolver-4tt5k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal","openshift-multus/multus-additional-cni-plugins-4pr7f","openshift-ovn-kubernetes/ovnkube-node-4vn9b","kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn","openshift-cluster-node-tuning-operator/tuned-fz6nx","openshift-image-registry/node-ca-2zcx5"] May 29 00:47:23.018892 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.018875 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jjscg" May 29 00:47:23.019973 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.019942 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-28 00:42:22 +0000 UTC" deadline="2028-02-19 06:20:24.228564026 +0000 UTC" May 29 00:47:23.019973 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.019968 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:23.019973 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.019971 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15149h33m1.208596129s" May 29 00:47:23.020104 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.020024 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:23.021362 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.021343 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:23.021464 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.021403 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:23.021527 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.021488 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 29 00:47:23.021527 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.021510 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 29 00:47:23.021603 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.021517 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 29 00:47:23.021603 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.021518 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fhdtw\"" May 29 00:47:23.021666 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.021623 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 29 00:47:23.022344 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.022318 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.023495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.023471 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-979kv" May 29 00:47:23.024087 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.023873 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.025096 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.024963 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 29 00:47:23.025247 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.025228 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4mft7\"" May 29 00:47:23.025618 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.025600 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 29 00:47:23.025701 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.025621 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 29 00:47:23.026027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.026011 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.026423 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.026406 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 29 00:47:23.026894 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.026801 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9jltw\"" May 29 00:47:23.027013 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.026801 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 29 00:47:23.027013 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.026950 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mxctp\"" May 29 00:47:23.027131 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.026861 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 29 00:47:23.027131 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.027039 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 29 00:47:23.027918 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.027900 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.028382 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.028350 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 29 00:47:23.028503 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.028413 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 29 00:47:23.028592 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.028556 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-whpjh\"" May 29 00:47:23.028688 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.028671 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 29 00:47:23.029256 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.029236 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.030359 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.030340 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 29 00:47:23.030563 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.030547 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.030961 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.030944 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 29 00:47:23.031057 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031032 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 29 00:47:23.031115 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031091 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8cks6\"" May 29 00:47:23.031199 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031171 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 29 00:47:23.031199 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031181 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 29 00:47:23.031337 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031278 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 29 00:47:23.031576 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031564 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 29 00:47:23.031627 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031600 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 29 00:47:23.031774 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031739 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 29 00:47:23.031952 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.031937 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.032086 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.032034 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nsgw6\"" May 29 00:47:23.032996 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.032979 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4qzgb\"" May 29 00:47:23.033081 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.033059 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 29 00:47:23.034236 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034203 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 29 00:47:23.034308 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034263 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cec494db-636c-4310-9706-a9ec296896ff-cni-binary-copy\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.034308 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034299 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k62cl\" (UniqueName: \"kubernetes.io/projected/cec494db-636c-4310-9706-a9ec296896ff-kube-api-access-k62cl\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.034412 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034327 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.034412 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034352 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-device-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.034412 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034379 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-iptables-alerter-script\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.034412 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034405 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-log-socket\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034414 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034429 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gq85h\"" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034444 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-cni-bin\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034461 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-kubelet\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034445 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034475 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034494 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b7c9\" (UniqueName: \"kubernetes.io/projected/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-kube-api-access-7b7c9\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034516 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31951da8-a753-4b89-9838-3f0c2c5505d1-hosts-file\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034537 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31951da8-a753-4b89-9838-3f0c2c5505d1-tmp-dir\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034558 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-kubelet\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.034605 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034589 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-slash\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034632 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94ea817f-e314-4f55-a064-dfafa9853ff7-agent-certs\") pod \"konnectivity-agent-979kv\" (UID: \"94ea817f-e314-4f55-a064-dfafa9853ff7\") " pod="kube-system/konnectivity-agent-979kv" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034666 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94ea817f-e314-4f55-a064-dfafa9853ff7-konnectivity-ca\") pod \"konnectivity-agent-979kv\" (UID: \"94ea817f-e314-4f55-a064-dfafa9853ff7\") " pod="kube-system/konnectivity-agent-979kv" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034691 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034700 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034733 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-ovn\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034787 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-conf-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034819 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cnibin\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034855 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxfnv\" (UniqueName: \"kubernetes.io/projected/12335c23-f1a8-4eae-9210-8801fa6ce0bd-kube-api-access-nxfnv\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034886 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-cni-netd\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034911 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-etc-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034932 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034949 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-cni-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034964 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-os-release\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.034982 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-hostroot\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.035027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035012 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035038 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dfdb2801-12f3-4d43-a796-c512dd303655-ovn-node-metrics-cert\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035068 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cec494db-636c-4310-9706-a9ec296896ff-multus-daemon-config\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035085 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035101 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-cni-bin\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035125 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-etc-kubernetes\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035141 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035157 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-system-cni-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035173 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-os-release\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035196 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035215 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-netns\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035232 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035252 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-run-netns\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035273 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-systemd\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035295 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-env-overrides\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035327 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-system-cni-dir\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.035530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035360 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035396 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-var-lib-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035437 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-ovnkube-script-lib\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035461 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcx6k\" (UniqueName: \"kubernetes.io/projected/dfdb2801-12f3-4d43-a796-c512dd303655-kube-api-access-jcx6k\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035500 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035560 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-socket-dir-parent\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035583 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-cni-multus\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035623 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-multus-certs\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035651 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035681 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r8z9\" (UniqueName: \"kubernetes.io/projected/31951da8-a753-4b89-9838-3f0c2c5505d1-kube-api-access-5r8z9\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035707 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035731 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-cnibin\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035771 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-k8s-cni-cncf-io\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035794 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhdfb\" (UniqueName: \"kubernetes.io/projected/f83593a1-25ed-4d6a-94f6-05e427aea6eb-kube-api-access-fhdfb\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035818 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v48cg\" (UniqueName: \"kubernetes.io/projected/f9e4aafe-dc70-4190-a323-d0f37c337d7a-kube-api-access-v48cg\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035843 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-host-slash\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.036172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035864 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-systemd-units\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.036639 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035889 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-run-ovn-kubernetes\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.036639 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035941 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-node-log\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.036639 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.035975 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-ovnkube-config\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.040230 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.040209 2551 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 29 00:47:23.056359 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.056339 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2mbxs" May 29 00:47:23.064178 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.064155 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2mbxs" May 29 00:47:23.070792 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.070731 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9682f407829b9ba775328d34f44bc9c.slice/crio-b627cf0963bf32b94e5d1f84a107e31d0b1e44aa1de4acb06e72479587811a9e WatchSource:0}: Error finding container b627cf0963bf32b94e5d1f84a107e31d0b1e44aa1de4acb06e72479587811a9e: Status 404 returned error can't find the container with id b627cf0963bf32b94e5d1f84a107e31d0b1e44aa1de4acb06e72479587811a9e May 29 00:47:23.071025 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.071002 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d23c050efd0f50b645c9f513ee415f.slice/crio-35c0c0620ca38480e0c2180e170fad880f13924a9b35bcbab8cbd06e78171c35 WatchSource:0}: Error finding container 35c0c0620ca38480e0c2180e170fad880f13924a9b35bcbab8cbd06e78171c35: Status 404 returned error can't find the container with id 35c0c0620ca38480e0c2180e170fad880f13924a9b35bcbab8cbd06e78171c35 May 29 00:47:23.074320 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.074304 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 00:47:23.129269 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.129253 2551 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 29 00:47:23.137140 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137115 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.137231 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137142 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-var-lib-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.137231 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137158 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-ovnkube-script-lib\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.137231 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137173 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jcx6k\" (UniqueName: \"kubernetes.io/projected/dfdb2801-12f3-4d43-a796-c512dd303655-kube-api-access-jcx6k\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.137231 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137196 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-tuned\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.137231 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137218 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.137467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137217 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-var-lib-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.137467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137243 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-socket-dir-parent\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137358 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-cni-multus\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137391 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-socket-dir-parent\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137400 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-multus-certs\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137427 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137428 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-cni-multus\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137478 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-multus-certs\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137480 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5r8z9\" (UniqueName: \"kubernetes.io/projected/31951da8-a753-4b89-9838-3f0c2c5505d1-kube-api-access-5r8z9\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137513 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137546 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-cnibin\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137571 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-k8s-cni-cncf-io\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137595 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fhdfb\" (UniqueName: \"kubernetes.io/projected/f83593a1-25ed-4d6a-94f6-05e427aea6eb-kube-api-access-fhdfb\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137621 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysconfig\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137633 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-cnibin\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137646 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpf8m\" (UniqueName: \"kubernetes.io/projected/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-kube-api-access-jpf8m\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137681 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v48cg\" (UniqueName: \"kubernetes.io/projected/f9e4aafe-dc70-4190-a323-d0f37c337d7a-kube-api-access-v48cg\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.137726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137705 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-host-slash\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137736 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-systemd-units\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137747 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-host-slash\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137775 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-run-ovn-kubernetes\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137787 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-ovnkube-script-lib\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137812 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-node-log\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137819 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-run-ovn-kubernetes\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137700 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-k8s-cni-cncf-io\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137839 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-ovnkube-config\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137863 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cec494db-636c-4310-9706-a9ec296896ff-cni-binary-copy\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137867 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-node-log\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137896 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k62cl\" (UniqueName: \"kubernetes.io/projected/cec494db-636c-4310-9706-a9ec296896ff-kube-api-access-k62cl\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137922 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137946 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-device-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137968 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-iptables-alerter-script\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137992 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-log-socket\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138019 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysctl-conf\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.138286 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138043 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-systemd\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138068 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-cni-bin\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138079 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-device-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138092 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-kubelet\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138097 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138119 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138130 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138155 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138168 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.137923 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-systemd-units\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138178 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7b7c9\" (UniqueName: \"kubernetes.io/projected/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-kube-api-access-7b7c9\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138192 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138209 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31951da8-a753-4b89-9838-3f0c2c5505d1-hosts-file\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138237 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-log-socket\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138260 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31951da8-a753-4b89-9838-3f0c2c5505d1-tmp-dir\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138279 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-cni-bin\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138285 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-kubelet\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138324 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31951da8-a753-4b89-9838-3f0c2c5505d1-hosts-file\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138329 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-slash\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138331 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-var-lib-kubelet\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138375 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94ea817f-e314-4f55-a064-dfafa9853ff7-agent-certs\") pod \"konnectivity-agent-979kv\" (UID: \"94ea817f-e314-4f55-a064-dfafa9853ff7\") " pod="kube-system/konnectivity-agent-979kv" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138384 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-ovnkube-config\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138405 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-kubelet\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138417 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94ea817f-e314-4f55-a064-dfafa9853ff7-konnectivity-ca\") pod \"konnectivity-agent-979kv\" (UID: \"94ea817f-e314-4f55-a064-dfafa9853ff7\") " pod="kube-system/konnectivity-agent-979kv" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138433 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-slash\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138442 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138489 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cec494db-636c-4310-9706-a9ec296896ff-cni-binary-copy\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138485 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-ovn\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138528 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-run\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138570 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq5ck\" (UniqueName: \"kubernetes.io/projected/f0790f93-cef8-451c-8dc7-358ce10becf4-kube-api-access-gq5ck\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138600 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-conf-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138603 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31951da8-a753-4b89-9838-3f0c2c5505d1-tmp-dir\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138618 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-ovn\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138624 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cnibin\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138662 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-cnibin\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.139943 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138662 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nxfnv\" (UniqueName: \"kubernetes.io/projected/12335c23-f1a8-4eae-9210-8801fa6ce0bd-kube-api-access-nxfnv\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138685 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-conf-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138695 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-cni-netd\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138729 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-modprobe-d\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138785 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-cni-netd\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138789 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-etc-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138827 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138861 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0790f93-cef8-451c-8dc7-358ce10becf4-serviceca\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138879 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-etc-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138886 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-cni-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138924 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-os-release\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138946 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-hostroot\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.138929 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138965 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138987 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dfdb2801-12f3-4d43-a796-c512dd303655-ovn-node-metrics-cert\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138992 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-os-release\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139010 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f83593a1-25ed-4d6a-94f6-05e427aea6eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.140631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.138947 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-multus-cni-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.139032 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs podName:12335c23-f1a8-4eae-9210-8801fa6ce0bd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:23.63899246 +0000 UTC m=+2.099195915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs") pod "network-metrics-daemon-px5qc" (UID: "12335c23-f1a8-4eae-9210-8801fa6ce0bd") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139059 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-hostroot\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139069 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cec494db-636c-4310-9706-a9ec296896ff-multus-daemon-config\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139098 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139123 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-cni-bin\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139139 2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139148 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-etc-kubernetes\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139179 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-openvswitch\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139193 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-cni-bin\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139194 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139219 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-etc-kubernetes\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139230 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139235 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-kubernetes\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139144 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139266 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-lib-modules\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139287 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-var-lib-kubelet\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.141242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139304 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-host\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139353 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0790f93-cef8-451c-8dc7-358ce10becf4-host\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139384 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-system-cni-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139434 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-os-release\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139462 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139475 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cec494db-636c-4310-9706-a9ec296896ff-multus-daemon-config\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139690 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysctl-d\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139695 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94ea817f-e314-4f55-a064-dfafa9853ff7-konnectivity-ca\") pod \"konnectivity-agent-979kv\" (UID: \"94ea817f-e314-4f55-a064-dfafa9853ff7\") " pod="kube-system/konnectivity-agent-979kv" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139714 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-tmp\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139638 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139521 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-system-cni-dir\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139548 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-iptables-alerter-script\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139544 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-os-release\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139779 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-netns\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139863 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139888 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-run-netns\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139811 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cec494db-636c-4310-9706-a9ec296896ff-host-run-netns\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.141699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139923 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-systemd\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139946 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-env-overrides\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139955 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9e4aafe-dc70-4190-a323-d0f37c337d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139972 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-sys\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139958 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-host-run-netns\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.139998 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-system-cni-dir\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.140003 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dfdb2801-12f3-4d43-a796-c512dd303655-run-systemd\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.140012 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" event={"ID":"b9682f407829b9ba775328d34f44bc9c","Type":"ContainerStarted","Data":"b627cf0963bf32b94e5d1f84a107e31d0b1e44aa1de4acb06e72479587811a9e"} May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.140077 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f83593a1-25ed-4d6a-94f6-05e427aea6eb-system-cni-dir\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.140302 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dfdb2801-12f3-4d43-a796-c512dd303655-env-overrides\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.142170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.141260 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" event={"ID":"c3d23c050efd0f50b645c9f513ee415f","Type":"ContainerStarted","Data":"35c0c0620ca38480e0c2180e170fad880f13924a9b35bcbab8cbd06e78171c35"} May 29 00:47:23.142441 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.142174 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dfdb2801-12f3-4d43-a796-c512dd303655-ovn-node-metrics-cert\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.142441 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.142233 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94ea817f-e314-4f55-a064-dfafa9853ff7-agent-certs\") pod \"konnectivity-agent-979kv\" (UID: \"94ea817f-e314-4f55-a064-dfafa9853ff7\") " pod="kube-system/konnectivity-agent-979kv" May 29 00:47:23.148595 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.148299 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 00:47:23.148595 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.148319 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 00:47:23.148595 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.148335 2551 projected.go:194] Error preparing data for projected volume kube-api-access-slkrw for pod openshift-network-diagnostics/network-check-target-6fjlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:23.148595 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.148407 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw podName:75386284-5f2d-431c-8d8d-19a580679c5b nodeName:}" failed. No retries permitted until 2026-05-29 00:47:23.648375088 +0000 UTC m=+2.108578515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-slkrw" (UniqueName: "kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw") pod "network-check-target-6fjlq" (UID: "75386284-5f2d-431c-8d8d-19a580679c5b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:23.149721 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.149701 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b7c9\" (UniqueName: \"kubernetes.io/projected/f16be3dd-4f67-4f0e-bff2-13cfdc3442c9-kube-api-access-7b7c9\") pod \"iptables-alerter-8x6h4\" (UID: \"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9\") " pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.149866 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.149737 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v48cg\" (UniqueName: \"kubernetes.io/projected/f9e4aafe-dc70-4190-a323-d0f37c337d7a-kube-api-access-v48cg\") pod \"aws-ebs-csi-driver-node-s9qhn\" (UID: \"f9e4aafe-dc70-4190-a323-d0f37c337d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.150108 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.150087 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhdfb\" (UniqueName: \"kubernetes.io/projected/f83593a1-25ed-4d6a-94f6-05e427aea6eb-kube-api-access-fhdfb\") pod \"multus-additional-cni-plugins-4pr7f\" (UID: \"f83593a1-25ed-4d6a-94f6-05e427aea6eb\") " pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.150559 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.150535 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k62cl\" (UniqueName: \"kubernetes.io/projected/cec494db-636c-4310-9706-a9ec296896ff-kube-api-access-k62cl\") pod \"multus-jjscg\" (UID: \"cec494db-636c-4310-9706-a9ec296896ff\") " pod="openshift-multus/multus-jjscg" May 29 00:47:23.150628 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.150558 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxfnv\" (UniqueName: \"kubernetes.io/projected/12335c23-f1a8-4eae-9210-8801fa6ce0bd-kube-api-access-nxfnv\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:23.150983 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.150966 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r8z9\" (UniqueName: \"kubernetes.io/projected/31951da8-a753-4b89-9838-3f0c2c5505d1-kube-api-access-5r8z9\") pod \"node-resolver-4tt5k\" (UID: \"31951da8-a753-4b89-9838-3f0c2c5505d1\") " pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.151024 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.150979 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcx6k\" (UniqueName: \"kubernetes.io/projected/dfdb2801-12f3-4d43-a796-c512dd303655-kube-api-access-jcx6k\") pod \"ovnkube-node-4vn9b\" (UID: \"dfdb2801-12f3-4d43-a796-c512dd303655\") " pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.241092 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241060 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-kubernetes\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241092 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241093 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-lib-modules\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241108 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-var-lib-kubelet\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241123 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-host\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241136 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0790f93-cef8-451c-8dc7-358ce10becf4-host\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241159 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysctl-d\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241182 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-tmp\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241197 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-kubernetes\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241202 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-lib-modules\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241180 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-var-lib-kubelet\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241207 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-sys\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241223 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0790f93-cef8-451c-8dc7-358ce10becf4-host\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241231 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-host\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241257 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-sys\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241260 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-tuned\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241260 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysctl-d\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241298 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysconfig\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241323 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jpf8m\" (UniqueName: \"kubernetes.io/projected/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-kube-api-access-jpf8m\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241362 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysctl-conf\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241386 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-systemd\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241364 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysconfig\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241431 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-run\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241450 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-sysctl-conf\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241469 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gq5ck\" (UniqueName: \"kubernetes.io/projected/f0790f93-cef8-451c-8dc7-358ce10becf4-kube-api-access-gq5ck\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241481 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-run\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241486 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-systemd\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241501 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-modprobe-d\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241538 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0790f93-cef8-451c-8dc7-358ce10becf4-serviceca\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241565 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-modprobe-d\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.241872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.241864 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0790f93-cef8-451c-8dc7-358ce10becf4-serviceca\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.242880 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.242860 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-tmp\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.243167 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.243153 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-etc-tuned\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.248353 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.248333 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq5ck\" (UniqueName: \"kubernetes.io/projected/f0790f93-cef8-451c-8dc7-358ce10becf4-kube-api-access-gq5ck\") pod \"node-ca-2zcx5\" (UID: \"f0790f93-cef8-451c-8dc7-358ce10becf4\") " pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.248419 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.248400 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpf8m\" (UniqueName: \"kubernetes.io/projected/5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0-kube-api-access-jpf8m\") pod \"tuned-fz6nx\" (UID: \"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0\") " pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.334249 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.334202 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jjscg" May 29 00:47:23.339916 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.339892 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec494db_636c_4310_9706_a9ec296896ff.slice/crio-bab3a5c086295fa1df8bef37dd8a42a5519e6aee2e19653ff639f8cc3a8247c2 WatchSource:0}: Error finding container bab3a5c086295fa1df8bef37dd8a42a5519e6aee2e19653ff639f8cc3a8247c2: Status 404 returned error can't find the container with id bab3a5c086295fa1df8bef37dd8a42a5519e6aee2e19653ff639f8cc3a8247c2 May 29 00:47:23.349879 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.349864 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8x6h4" May 29 00:47:23.354694 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.354675 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16be3dd_4f67_4f0e_bff2_13cfdc3442c9.slice/crio-79ff6a5bc5ab6f5923f46fcf71ca788b9123e2d00df61cdc599f1e0e0a09a2bf WatchSource:0}: Error finding container 79ff6a5bc5ab6f5923f46fcf71ca788b9123e2d00df61cdc599f1e0e0a09a2bf: Status 404 returned error can't find the container with id 79ff6a5bc5ab6f5923f46fcf71ca788b9123e2d00df61cdc599f1e0e0a09a2bf May 29 00:47:23.377331 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.377311 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-979kv" May 29 00:47:23.381831 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.381797 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4tt5k" May 29 00:47:23.385072 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.385038 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ea817f_e314_4f55_a064_dfafa9853ff7.slice/crio-2a249a246a943b15dd3805e192baeb3abcb2e2fbb390738b2d236e510f9907b3 WatchSource:0}: Error finding container 2a249a246a943b15dd3805e192baeb3abcb2e2fbb390738b2d236e510f9907b3: Status 404 returned error can't find the container with id 2a249a246a943b15dd3805e192baeb3abcb2e2fbb390738b2d236e510f9907b3 May 29 00:47:23.388773 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.388728 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31951da8_a753_4b89_9838_3f0c2c5505d1.slice/crio-60281220deaa010a4347c38a25df8970f324c92cd7f2e9956e35291763c2119c WatchSource:0}: Error finding container 60281220deaa010a4347c38a25df8970f324c92cd7f2e9956e35291763c2119c: Status 404 returned error can't find the container with id 60281220deaa010a4347c38a25df8970f324c92cd7f2e9956e35291763c2119c May 29 00:47:23.388866 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.388787 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" May 29 00:47:23.394495 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.394477 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:23.396558 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.396535 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83593a1_25ed_4d6a_94f6_05e427aea6eb.slice/crio-3dcf2a03759f91dd1e1c30f908933e25c7aaa3f098cd2b93d1e13efecd74534c WatchSource:0}: Error finding container 3dcf2a03759f91dd1e1c30f908933e25c7aaa3f098cd2b93d1e13efecd74534c: Status 404 returned error can't find the container with id 3dcf2a03759f91dd1e1c30f908933e25c7aaa3f098cd2b93d1e13efecd74534c May 29 00:47:23.400089 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.400068 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" May 29 00:47:23.401788 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.401741 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfdb2801_12f3_4d43_a796_c512dd303655.slice/crio-316a49bcd786746a8b25f9e010cb42ef6749de363d1a4ea3f010d4c5454833bd WatchSource:0}: Error finding container 316a49bcd786746a8b25f9e010cb42ef6749de363d1a4ea3f010d4c5454833bd: Status 404 returned error can't find the container with id 316a49bcd786746a8b25f9e010cb42ef6749de363d1a4ea3f010d4c5454833bd May 29 00:47:23.404861 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.404816 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" May 29 00:47:23.408320 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.408294 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e4aafe_dc70_4190_a323_d0f37c337d7a.slice/crio-f8bd65323f9801e552a7425fb132b9dd3e57a7e773f5e5269b383c0024c60336 WatchSource:0}: Error finding container f8bd65323f9801e552a7425fb132b9dd3e57a7e773f5e5269b383c0024c60336: Status 404 returned error can't find the container with id f8bd65323f9801e552a7425fb132b9dd3e57a7e773f5e5269b383c0024c60336 May 29 00:47:23.409538 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.409498 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2zcx5" May 29 00:47:23.414186 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.414163 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc351b6_5e0a_4d78_81c7_dff79b5d7bd0.slice/crio-df64f540205afc149d65341ea2ed08d2593d8a4890ec8b1f9e1db3aad39e88af WatchSource:0}: Error finding container df64f540205afc149d65341ea2ed08d2593d8a4890ec8b1f9e1db3aad39e88af: Status 404 returned error can't find the container with id df64f540205afc149d65341ea2ed08d2593d8a4890ec8b1f9e1db3aad39e88af May 29 00:47:23.417375 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:47:23.417322 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0790f93_cef8_451c_8dc7_358ce10becf4.slice/crio-6a5e9aaba1b775878e92a748a33cf9122d61dce89fc0065690b3b63e91ae000b WatchSource:0}: Error finding container 6a5e9aaba1b775878e92a748a33cf9122d61dce89fc0065690b3b63e91ae000b: Status 404 returned error can't find the container with id 6a5e9aaba1b775878e92a748a33cf9122d61dce89fc0065690b3b63e91ae000b May 29 00:47:23.644303 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.644228 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:23.644444 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.644343 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:23.644444 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.644404 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs podName:12335c23-f1a8-4eae-9210-8801fa6ce0bd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:24.6443869 +0000 UTC m=+3.104590341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs") pod "network-metrics-daemon-px5qc" (UID: "12335c23-f1a8-4eae-9210-8801fa6ce0bd") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:23.746665 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:23.746633 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:23.746835 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.746818 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 00:47:23.746891 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.746839 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 00:47:23.746891 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.746851 2551 projected.go:194] Error preparing data for projected volume kube-api-access-slkrw for pod openshift-network-diagnostics/network-check-target-6fjlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:23.746983 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:23.746903 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw podName:75386284-5f2d-431c-8d8d-19a580679c5b nodeName:}" failed. No retries permitted until 2026-05-29 00:47:24.746884087 +0000 UTC m=+3.207087518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-slkrw" (UniqueName: "kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw") pod "network-check-target-6fjlq" (UID: "75386284-5f2d-431c-8d8d-19a580679c5b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:24.065216 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.065056 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 00:42:23 +0000 UTC" deadline="2028-02-28 21:02:11.686531532 +0000 UTC" May 29 00:47:24.065216 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.065090 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15380h14m47.621444462s" May 29 00:47:24.155694 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.155611 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" event={"ID":"f83593a1-25ed-4d6a-94f6-05e427aea6eb","Type":"ContainerStarted","Data":"3dcf2a03759f91dd1e1c30f908933e25c7aaa3f098cd2b93d1e13efecd74534c"} May 29 00:47:24.158549 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.158518 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-979kv" event={"ID":"94ea817f-e314-4f55-a064-dfafa9853ff7","Type":"ContainerStarted","Data":"2a249a246a943b15dd3805e192baeb3abcb2e2fbb390738b2d236e510f9907b3"} May 29 00:47:24.162274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.162245 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jjscg" event={"ID":"cec494db-636c-4310-9706-a9ec296896ff","Type":"ContainerStarted","Data":"bab3a5c086295fa1df8bef37dd8a42a5519e6aee2e19653ff639f8cc3a8247c2"} May 29 00:47:24.166472 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.166445 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" event={"ID":"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0","Type":"ContainerStarted","Data":"df64f540205afc149d65341ea2ed08d2593d8a4890ec8b1f9e1db3aad39e88af"} May 29 00:47:24.177920 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.177894 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" event={"ID":"f9e4aafe-dc70-4190-a323-d0f37c337d7a","Type":"ContainerStarted","Data":"f8bd65323f9801e552a7425fb132b9dd3e57a7e773f5e5269b383c0024c60336"} May 29 00:47:24.183744 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.183711 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"316a49bcd786746a8b25f9e010cb42ef6749de363d1a4ea3f010d4c5454833bd"} May 29 00:47:24.192392 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.192360 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4tt5k" event={"ID":"31951da8-a753-4b89-9838-3f0c2c5505d1","Type":"ContainerStarted","Data":"60281220deaa010a4347c38a25df8970f324c92cd7f2e9956e35291763c2119c"} May 29 00:47:24.202735 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.202683 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8x6h4" event={"ID":"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9","Type":"ContainerStarted","Data":"79ff6a5bc5ab6f5923f46fcf71ca788b9123e2d00df61cdc599f1e0e0a09a2bf"} May 29 00:47:24.209645 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.209616 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2zcx5" event={"ID":"f0790f93-cef8-451c-8dc7-358ce10becf4","Type":"ContainerStarted","Data":"6a5e9aaba1b775878e92a748a33cf9122d61dce89fc0065690b3b63e91ae000b"} May 29 00:47:24.240880 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.240858 2551 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 29 00:47:24.454488 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.454205 2551 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 29 00:47:24.654020 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.653987 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:24.654213 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:24.654120 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:24.654213 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:24.654174 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs podName:12335c23-f1a8-4eae-9210-8801fa6ce0bd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:26.654158373 +0000 UTC m=+5.114361814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs") pod "network-metrics-daemon-px5qc" (UID: "12335c23-f1a8-4eae-9210-8801fa6ce0bd") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:24.754691 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:24.754614 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:24.754851 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:24.754791 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 00:47:24.754851 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:24.754809 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 00:47:24.754851 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:24.754821 2551 projected.go:194] Error preparing data for projected volume kube-api-access-slkrw for pod openshift-network-diagnostics/network-check-target-6fjlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:24.755009 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:24.754871 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw podName:75386284-5f2d-431c-8d8d-19a580679c5b nodeName:}" failed. No retries permitted until 2026-05-29 00:47:26.754853766 +0000 UTC m=+5.215057194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-slkrw" (UniqueName: "kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw") pod "network-check-target-6fjlq" (UID: "75386284-5f2d-431c-8d8d-19a580679c5b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:25.065649 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:25.065551 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 00:42:23 +0000 UTC" deadline="2027-11-07 04:08:40.982076311 +0000 UTC" May 29 00:47:25.065649 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:25.065593 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12651h21m15.916487462s" May 29 00:47:25.137667 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:25.136905 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:25.137667 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:25.137019 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:25.137667 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:25.137533 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:25.137667 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:25.137625 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:25.791699 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:25.791255 2551 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 29 00:47:26.098859 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:26.098835 2551 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 29 00:47:26.218103 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:26.217508 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" event={"ID":"c3d23c050efd0f50b645c9f513ee415f","Type":"ContainerStarted","Data":"33c474f20e51288d545731089d115ce59d5133e359201276cf2ad0948d23e6fd"} May 29 00:47:26.668491 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:26.668456 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:26.668644 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:26.668602 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:26.668713 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:26.668670 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs podName:12335c23-f1a8-4eae-9210-8801fa6ce0bd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:30.668647233 +0000 UTC m=+9.128850662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs") pod "network-metrics-daemon-px5qc" (UID: "12335c23-f1a8-4eae-9210-8801fa6ce0bd") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:26.769381 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:26.769330 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:26.769556 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:26.769522 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 00:47:26.769556 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:26.769543 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 00:47:26.769556 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:26.769555 2551 projected.go:194] Error preparing data for projected volume kube-api-access-slkrw for pod openshift-network-diagnostics/network-check-target-6fjlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:26.769713 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:26.769614 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw podName:75386284-5f2d-431c-8d8d-19a580679c5b nodeName:}" failed. No retries permitted until 2026-05-29 00:47:30.769594568 +0000 UTC m=+9.229797999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-slkrw" (UniqueName: "kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw") pod "network-check-target-6fjlq" (UID: "75386284-5f2d-431c-8d8d-19a580679c5b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:27.136514 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:27.136481 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:27.136959 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:27.136644 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:27.136959 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:27.136727 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:27.136959 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:27.136827 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:27.221072 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:27.221040 2551 generic.go:358] "Generic (PLEG): container finished" podID="c3d23c050efd0f50b645c9f513ee415f" containerID="33c474f20e51288d545731089d115ce59d5133e359201276cf2ad0948d23e6fd" exitCode=0 May 29 00:47:27.221230 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:27.221088 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" event={"ID":"c3d23c050efd0f50b645c9f513ee415f","Type":"ContainerDied","Data":"33c474f20e51288d545731089d115ce59d5133e359201276cf2ad0948d23e6fd"} May 29 00:47:29.137407 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:29.137353 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:29.137407 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:29.137388 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:29.137901 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:29.137474 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:29.137901 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:29.137609 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:30.701393 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:30.701359 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:30.701946 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:30.701469 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:30.701946 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:30.701544 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs podName:12335c23-f1a8-4eae-9210-8801fa6ce0bd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:38.701523452 +0000 UTC m=+17.161726894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs") pod "network-metrics-daemon-px5qc" (UID: "12335c23-f1a8-4eae-9210-8801fa6ce0bd") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:30.802370 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:30.801805 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:30.802370 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:30.801973 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 00:47:30.802370 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:30.801990 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 00:47:30.802370 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:30.802002 2551 projected.go:194] Error preparing data for projected volume kube-api-access-slkrw for pod openshift-network-diagnostics/network-check-target-6fjlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:30.802370 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:30.802053 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw podName:75386284-5f2d-431c-8d8d-19a580679c5b nodeName:}" failed. No retries permitted until 2026-05-29 00:47:38.802035178 +0000 UTC m=+17.262238613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-slkrw" (UniqueName: "kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw") pod "network-check-target-6fjlq" (UID: "75386284-5f2d-431c-8d8d-19a580679c5b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:31.137358 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:31.137329 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:31.137519 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:31.137340 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:31.137519 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:31.137449 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:31.137696 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:31.137545 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:33.137505 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:33.137470 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:33.137955 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:33.137470 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:33.137955 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:33.137599 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:33.137955 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:33.137672 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:35.136496 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:35.136461 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:35.136496 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:35.136478 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:35.136939 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:35.136569 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:35.136939 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:35.136660 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:37.137172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:37.137106 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:37.137519 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:37.137110 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:37.137519 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:37.137201 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:37.137519 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:37.137304 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:38.768548 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:38.768513 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:38.768983 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:38.768669 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:38.768983 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:38.768741 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs podName:12335c23-f1a8-4eae-9210-8801fa6ce0bd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:54.768719387 +0000 UTC m=+33.228922822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs") pod "network-metrics-daemon-px5qc" (UID: "12335c23-f1a8-4eae-9210-8801fa6ce0bd") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:38.869483 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:38.869452 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:38.869651 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:38.869631 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 00:47:38.869703 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:38.869657 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 00:47:38.869703 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:38.869671 2551 projected.go:194] Error preparing data for projected volume kube-api-access-slkrw for pod openshift-network-diagnostics/network-check-target-6fjlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:38.869802 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:38.869728 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw podName:75386284-5f2d-431c-8d8d-19a580679c5b nodeName:}" failed. No retries permitted until 2026-05-29 00:47:54.869709773 +0000 UTC m=+33.329913208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-slkrw" (UniqueName: "kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw") pod "network-check-target-6fjlq" (UID: "75386284-5f2d-431c-8d8d-19a580679c5b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:39.136646 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:39.136573 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:39.136817 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:39.136573 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:39.136817 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:39.136697 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:39.136920 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:39.136811 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:41.137069 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:41.137041 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:41.137442 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:41.137041 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:41.137442 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:41.137157 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:41.137442 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:41.137231 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:42.241322 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.241009 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" event={"ID":"5fc351b6-5e0a-4d78-81c7-dff79b5d7bd0","Type":"ContainerStarted","Data":"1d2b8cd66e2cd72b292635b751b05a36a80471d7c1d976cc25924132462623d7"} May 29 00:47:42.242070 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.242051 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" event={"ID":"f9e4aafe-dc70-4190-a323-d0f37c337d7a","Type":"ContainerStarted","Data":"f5324820dbed75a1288d689d3e28eec66ec28212351f4f31e8d9c925ae1baa84"} May 29 00:47:42.243984 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.243961 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"e6a8bfb496811bbf36b09af76e20ed6e9a546f6b58445dfcd35927285407078b"} May 29 00:47:42.244060 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.243990 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"ab2d286ff4b9440a929ce134d80696fb3d0ac8c1243548f8aeb38946f5b8077f"} May 29 00:47:42.244060 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.244003 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"027c3dd05455436facbda0703f374d8645073c5a0fec15f079c3da82a7c91033"} May 29 00:47:42.244060 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.244015 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"57ec932f4729789e83b4d7a3ebf17cb8cffaf1da0ee739b85b3c6ddbf10de8a7"} May 29 00:47:42.244060 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.244025 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"cd1861520b9d8d44c070e2de229705673d00028588185b1cb297a86392bdf69f"} May 29 00:47:42.245016 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.245000 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4tt5k" event={"ID":"31951da8-a753-4b89-9838-3f0c2c5505d1","Type":"ContainerStarted","Data":"62c576c74c1ba46ae50d9f24ea78e8fed2296d30b11d747afcddb6649dfccf87"} May 29 00:47:42.246020 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.245978 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" event={"ID":"b9682f407829b9ba775328d34f44bc9c","Type":"ContainerStarted","Data":"70922d43457b763c3d2d0629a0b32debcac447f240a694039e2f4c20814631ed"} May 29 00:47:42.247282 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.247258 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" event={"ID":"c3d23c050efd0f50b645c9f513ee415f","Type":"ContainerStarted","Data":"9f6bc0caec8c8305a24b899e8187cda11a8c43500d8f7664900d74dc94ff09b2"} May 29 00:47:42.248345 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.248321 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2zcx5" event={"ID":"f0790f93-cef8-451c-8dc7-358ce10becf4","Type":"ContainerStarted","Data":"7958656e273fd3f1f17a848198f3bab4c9e43a312ef196bf8fd409b6cf160fa5"} May 29 00:47:42.251611 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.251590 2551 generic.go:358] "Generic (PLEG): container finished" podID="f83593a1-25ed-4d6a-94f6-05e427aea6eb" containerID="4e3432e99a94d1875a991c296edf41275a3068dffc220e7084ec1aa9aa2901a1" exitCode=0 May 29 00:47:42.251689 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.251654 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" event={"ID":"f83593a1-25ed-4d6a-94f6-05e427aea6eb","Type":"ContainerDied","Data":"4e3432e99a94d1875a991c296edf41275a3068dffc220e7084ec1aa9aa2901a1"} May 29 00:47:42.252914 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.252846 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-979kv" event={"ID":"94ea817f-e314-4f55-a064-dfafa9853ff7","Type":"ContainerStarted","Data":"4f09348d94a286bd93096dfe088c231bbdfa0d8798b230309d7e9f462462b7df"} May 29 00:47:42.253989 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.253970 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jjscg" event={"ID":"cec494db-636c-4310-9706-a9ec296896ff","Type":"ContainerStarted","Data":"fbb47e0397f36c5eae6330c73bba10495adf472ff40e2cf15a40ddefc73f62cb"} May 29 00:47:42.277714 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.277669 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-236.ec2.internal" podStartSLOduration=20.277656079 podStartE2EDuration="20.277656079s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 00:47:42.261182837 +0000 UTC m=+20.721386285" watchObservedRunningTime="2026-05-29 00:47:42.277656079 +0000 UTC m=+20.737859529" May 29 00:47:42.277887 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.277858 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fz6nx" podStartSLOduration=2.044140809 podStartE2EDuration="20.277849103s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.415913337 +0000 UTC m=+1.876116769" lastFinishedPulling="2026-05-29 00:47:41.649621627 +0000 UTC m=+20.109825063" observedRunningTime="2026-05-29 00:47:42.277112595 +0000 UTC m=+20.737316044" watchObservedRunningTime="2026-05-29 00:47:42.277849103 +0000 UTC m=+20.738052552" May 29 00:47:42.291096 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.291067 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-236.ec2.internal" podStartSLOduration=20.291058516 podStartE2EDuration="20.291058516s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 00:47:42.290784723 +0000 UTC m=+20.750988168" watchObservedRunningTime="2026-05-29 00:47:42.291058516 +0000 UTC m=+20.751261965" May 29 00:47:42.307531 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.307467 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-jjscg" podStartSLOduration=1.992753658 podStartE2EDuration="20.307455221s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.341212849 +0000 UTC m=+1.801416277" lastFinishedPulling="2026-05-29 00:47:41.655914408 +0000 UTC m=+20.116117840" observedRunningTime="2026-05-29 00:47:42.306907961 +0000 UTC m=+20.767111421" watchObservedRunningTime="2026-05-29 00:47:42.307455221 +0000 UTC m=+20.767658672" May 29 00:47:42.321408 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.321380 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-979kv" podStartSLOduration=6.83298545 podStartE2EDuration="20.321370964s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.386708186 +0000 UTC m=+1.846911613" lastFinishedPulling="2026-05-29 00:47:36.875093689 +0000 UTC m=+15.335297127" observedRunningTime="2026-05-29 00:47:42.321056644 +0000 UTC m=+20.781260096" watchObservedRunningTime="2026-05-29 00:47:42.321370964 +0000 UTC m=+20.781574412" May 29 00:47:42.354481 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:42.354444 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-4tt5k" podStartSLOduration=2.097263365 podStartE2EDuration="20.354434441s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.391013352 +0000 UTC m=+1.851216783" lastFinishedPulling="2026-05-29 00:47:41.648184418 +0000 UTC m=+20.108387859" observedRunningTime="2026-05-29 00:47:42.334813755 +0000 UTC m=+20.795017214" watchObservedRunningTime="2026-05-29 00:47:42.354434441 +0000 UTC m=+20.814637889" May 29 00:47:43.136621 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:43.136593 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:43.136814 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:43.136635 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:43.136814 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:43.136711 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:43.136814 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:43.136802 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:43.258769 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:43.258719 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"037c6923f13a61b4d270a9e0035d876df120b29eda9fa4b0aac9221435794079"} May 29 00:47:43.260194 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:43.260170 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8x6h4" event={"ID":"f16be3dd-4f67-4f0e-bff2-13cfdc3442c9","Type":"ContainerStarted","Data":"a6eb189eb470c97f0815e9811d5c80710c0532cd8e71c153c436be97873c1d4f"} May 29 00:47:43.275000 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:43.274960 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2zcx5" podStartSLOduration=3.047352498 podStartE2EDuration="21.274946036s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.418707422 +0000 UTC m=+1.878910849" lastFinishedPulling="2026-05-29 00:47:41.646300954 +0000 UTC m=+20.106504387" observedRunningTime="2026-05-29 00:47:42.368045485 +0000 UTC m=+20.828248934" watchObservedRunningTime="2026-05-29 00:47:43.274946036 +0000 UTC m=+21.735149484" May 29 00:47:43.275247 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:43.275216 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8x6h4" podStartSLOduration=2.984889225 podStartE2EDuration="21.27520776s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.356107513 +0000 UTC m=+1.816310941" lastFinishedPulling="2026-05-29 00:47:41.646426039 +0000 UTC m=+20.106629476" observedRunningTime="2026-05-29 00:47:43.274694024 +0000 UTC m=+21.734897483" watchObservedRunningTime="2026-05-29 00:47:43.27520776 +0000 UTC m=+21.735411210" May 29 00:47:43.692572 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:43.692547 2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 29 00:47:44.101693 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:44.101581 2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-29T00:47:43.692568724Z","UUID":"d9d87a55-6df0-472f-868f-4bd81cce0617","Handler":null,"Name":"","Endpoint":""} May 29 00:47:44.104164 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:44.104140 2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 29 00:47:44.104164 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:44.104173 2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 29 00:47:44.146093 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:44.146070 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-979kv" May 29 00:47:44.146628 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:44.146612 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-979kv" May 29 00:47:44.264003 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:44.263974 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" event={"ID":"f9e4aafe-dc70-4190-a323-d0f37c337d7a","Type":"ContainerStarted","Data":"402a90deefc11250eea2ecd6bcefe4013f3b8faa9e0dae53b80c482b0afa74ff"} May 29 00:47:44.264396 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:44.264323 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-979kv" May 29 00:47:44.264768 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:44.264735 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-979kv" May 29 00:47:45.137028 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:45.136870 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:45.137173 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:45.136876 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:45.137173 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:45.137121 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:45.137275 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:45.137197 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:45.267909 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:45.267874 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" event={"ID":"f9e4aafe-dc70-4190-a323-d0f37c337d7a","Type":"ContainerStarted","Data":"ec1af642795a77b0a1c98387ee3929e3ebb37af117054a843431869228cf97b0"} May 29 00:47:45.271017 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:45.270992 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"ab4d17addce786a76d8bf2bac73c2413271abda16d1a86ec80a6c28fa2f85ac7"} May 29 00:47:45.288319 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:45.288282 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s9qhn" podStartSLOduration=1.790345185 podStartE2EDuration="23.288266376s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.410716474 +0000 UTC m=+1.870919908" lastFinishedPulling="2026-05-29 00:47:44.908637657 +0000 UTC m=+23.368841099" observedRunningTime="2026-05-29 00:47:45.287958151 +0000 UTC m=+23.748161604" watchObservedRunningTime="2026-05-29 00:47:45.288266376 +0000 UTC m=+23.748469824" May 29 00:47:47.137338 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.137166 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:47.137932 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.137166 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:47.137932 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:47.137413 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:47.137932 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:47.137489 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:47.276108 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.276080 2551 generic.go:358] "Generic (PLEG): container finished" podID="f83593a1-25ed-4d6a-94f6-05e427aea6eb" containerID="cb1bb46a520be1c99a3105243752d217f6e3e388d747b59d758ec24d958daded" exitCode=0 May 29 00:47:47.276214 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.276171 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" event={"ID":"f83593a1-25ed-4d6a-94f6-05e427aea6eb","Type":"ContainerDied","Data":"cb1bb46a520be1c99a3105243752d217f6e3e388d747b59d758ec24d958daded"} May 29 00:47:47.279396 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.279374 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" event={"ID":"dfdb2801-12f3-4d43-a796-c512dd303655","Type":"ContainerStarted","Data":"56b0ead68251fdc03ec996d3ac9fc0fbc0d604f4f1e88db00e6c3e9d006cd9fc"} May 29 00:47:47.279611 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.279594 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:47.279692 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.279616 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:47.294946 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.294925 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:47.320148 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:47.320106 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" podStartSLOduration=6.920737216 podStartE2EDuration="25.320095368s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.403422405 +0000 UTC m=+1.863625833" lastFinishedPulling="2026-05-29 00:47:41.802780558 +0000 UTC m=+20.262983985" observedRunningTime="2026-05-29 00:47:47.318401865 +0000 UTC m=+25.778605314" watchObservedRunningTime="2026-05-29 00:47:47.320095368 +0000 UTC m=+25.780298815" May 29 00:47:48.282725 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:48.282691 2551 generic.go:358] "Generic (PLEG): container finished" podID="f83593a1-25ed-4d6a-94f6-05e427aea6eb" containerID="4fdee340ce6672001b4ed5455a929db1cd3a4deb0fce6fc4e9b137567518faed" exitCode=0 May 29 00:47:48.283168 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:48.282781 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" event={"ID":"f83593a1-25ed-4d6a-94f6-05e427aea6eb","Type":"ContainerDied","Data":"4fdee340ce6672001b4ed5455a929db1cd3a4deb0fce6fc4e9b137567518faed"} May 29 00:47:48.283444 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:48.283400 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:48.297787 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:48.297766 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:47:48.712642 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:48.712588 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6fjlq"] May 29 00:47:48.712798 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:48.712713 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:48.712862 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:48.712821 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:48.713235 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:48.713205 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-px5qc"] May 29 00:47:48.713346 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:48.713316 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:48.713433 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:48.713409 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:49.286928 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:49.286724 2551 generic.go:358] "Generic (PLEG): container finished" podID="f83593a1-25ed-4d6a-94f6-05e427aea6eb" containerID="9a0825d930bff813f6200c6e8b36b669c3807316f3e8bd7b8954e81e5b667dd0" exitCode=0 May 29 00:47:49.287258 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:49.286811 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" event={"ID":"f83593a1-25ed-4d6a-94f6-05e427aea6eb","Type":"ContainerDied","Data":"9a0825d930bff813f6200c6e8b36b669c3807316f3e8bd7b8954e81e5b667dd0"} May 29 00:47:50.137371 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:50.137342 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:50.137520 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:50.137406 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:50.137583 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:50.137514 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:50.137675 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:50.137649 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:52.138076 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:52.138044 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:52.138593 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:52.138242 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:52.138593 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:52.138348 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:52.138593 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:52.138195 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:54.136862 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.136835 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:54.137471 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.136966 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-px5qc" podUID="12335c23-f1a8-4eae-9210-8801fa6ce0bd" May 29 00:47:54.137471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.137029 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:54.137471 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.137110 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6fjlq" podUID="75386284-5f2d-431c-8d8d-19a580679c5b" May 29 00:47:54.386107 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.386083 2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-236.ec2.internal" event="NodeReady" May 29 00:47:54.386277 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.386189 2551 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 29 00:47:54.427856 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.424884 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n5lpq"] May 29 00:47:54.458627 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.458601 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n5szr"] May 29 00:47:54.458794 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.458764 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.461682 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.461629 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 29 00:47:54.461682 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.461648 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7vx5w\"" May 29 00:47:54.461854 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.461776 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 29 00:47:54.472744 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.472725 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n5lpq"] May 29 00:47:54.472744 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.472762 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n5szr"] May 29 00:47:54.472919 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.472859 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:54.475940 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.475705 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 29 00:47:54.475940 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.475705 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 29 00:47:54.475940 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.475837 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nghjs\"" May 29 00:47:54.475940 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.475917 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 29 00:47:54.586432 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.586395 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.586601 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.586448 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-config-volume\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.586601 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.586480 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:54.586601 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.586514 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv99g\" (UniqueName: \"kubernetes.io/projected/bec90e87-7a28-4f14-83c6-61def84dbed9-kube-api-access-rv99g\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:54.586601 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.586550 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbxrw\" (UniqueName: \"kubernetes.io/projected/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-kube-api-access-vbxrw\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.586799 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.586609 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-tmp-dir\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.687628 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.687539 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vbxrw\" (UniqueName: \"kubernetes.io/projected/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-kube-api-access-vbxrw\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.687628 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.687603 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-tmp-dir\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.687865 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.687653 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.687865 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.687678 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-config-volume\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.687865 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.687702 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:54.687865 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.687726 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rv99g\" (UniqueName: \"kubernetes.io/projected/bec90e87-7a28-4f14-83c6-61def84dbed9-kube-api-access-rv99g\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:54.687865 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.687803 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 00:47:54.687865 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.687851 2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 00:47:54.688170 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.687892 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls podName:a0aa4161-0aa4-4e61-b55e-d737c3bfaecd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:55.187870522 +0000 UTC m=+33.648073953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls") pod "dns-default-n5lpq" (UID: "a0aa4161-0aa4-4e61-b55e-d737c3bfaecd") : secret "dns-default-metrics-tls" not found May 29 00:47:54.688170 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.687910 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert podName:bec90e87-7a28-4f14-83c6-61def84dbed9 nodeName:}" failed. No retries permitted until 2026-05-29 00:47:55.187901867 +0000 UTC m=+33.648105298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert") pod "ingress-canary-n5szr" (UID: "bec90e87-7a28-4f14-83c6-61def84dbed9") : secret "canary-serving-cert" not found May 29 00:47:54.688170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.688165 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-tmp-dir\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.688404 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.688383 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-config-volume\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.700853 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.700819 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbxrw\" (UniqueName: \"kubernetes.io/projected/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-kube-api-access-vbxrw\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:54.700991 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.700883 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv99g\" (UniqueName: \"kubernetes.io/projected/bec90e87-7a28-4f14-83c6-61def84dbed9-kube-api-access-rv99g\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:54.788160 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.788129 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:54.788279 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.788219 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:54.788279 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.788274 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs podName:12335c23-f1a8-4eae-9210-8801fa6ce0bd nodeName:}" failed. No retries permitted until 2026-05-29 00:48:26.788256806 +0000 UTC m=+65.248460237 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs") pod "network-metrics-daemon-px5qc" (UID: "12335c23-f1a8-4eae-9210-8801fa6ce0bd") : object "openshift-multus"/"metrics-daemon-secret" not registered May 29 00:47:54.889460 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:54.889435 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:54.889630 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.889609 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 29 00:47:54.889630 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.889629 2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 29 00:47:54.889830 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.889638 2551 projected.go:194] Error preparing data for projected volume kube-api-access-slkrw for pod openshift-network-diagnostics/network-check-target-6fjlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:54.889830 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:54.889689 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw podName:75386284-5f2d-431c-8d8d-19a580679c5b nodeName:}" failed. No retries permitted until 2026-05-29 00:48:26.889670297 +0000 UTC m=+65.349873727 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-slkrw" (UniqueName: "kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw") pod "network-check-target-6fjlq" (UID: "75386284-5f2d-431c-8d8d-19a580679c5b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 29 00:47:55.191413 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:55.191387 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:55.191914 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:55.191420 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:55.191914 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:55.191515 2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 00:47:55.191914 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:55.191554 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert podName:bec90e87-7a28-4f14-83c6-61def84dbed9 nodeName:}" failed. No retries permitted until 2026-05-29 00:47:56.191541726 +0000 UTC m=+34.651745157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert") pod "ingress-canary-n5szr" (UID: "bec90e87-7a28-4f14-83c6-61def84dbed9") : secret "canary-serving-cert" not found May 29 00:47:55.191914 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:55.191515 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 00:47:55.191914 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:55.191644 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls podName:a0aa4161-0aa4-4e61-b55e-d737c3bfaecd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:56.191629325 +0000 UTC m=+34.651832752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls") pod "dns-default-n5lpq" (UID: "a0aa4161-0aa4-4e61-b55e-d737c3bfaecd") : secret "dns-default-metrics-tls" not found May 29 00:47:56.137269 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.137237 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:47:56.137434 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.137237 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:47:56.140262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.140244 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 29 00:47:56.140382 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.140322 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 29 00:47:56.141715 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.141701 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c2thf\"" May 29 00:47:56.141797 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.141780 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 29 00:47:56.141842 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.141807 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rlrhs\"" May 29 00:47:56.199622 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.199603 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:56.199908 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.199632 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:56.199908 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:56.199728 2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 00:47:56.199908 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:56.199731 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 00:47:56.199908 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:56.199785 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert podName:bec90e87-7a28-4f14-83c6-61def84dbed9 nodeName:}" failed. No retries permitted until 2026-05-29 00:47:58.199771754 +0000 UTC m=+36.659975182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert") pod "ingress-canary-n5szr" (UID: "bec90e87-7a28-4f14-83c6-61def84dbed9") : secret "canary-serving-cert" not found May 29 00:47:56.199908 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:56.199797 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls podName:a0aa4161-0aa4-4e61-b55e-d737c3bfaecd nodeName:}" failed. No retries permitted until 2026-05-29 00:47:58.199791947 +0000 UTC m=+36.659995374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls") pod "dns-default-n5lpq" (UID: "a0aa4161-0aa4-4e61-b55e-d737c3bfaecd") : secret "dns-default-metrics-tls" not found May 29 00:47:56.301570 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.301551 2551 generic.go:358] "Generic (PLEG): container finished" podID="f83593a1-25ed-4d6a-94f6-05e427aea6eb" containerID="d18793b38fd393bd49a4d9bc1d7624dbdf61353e715ef4d9a19484afcb60d6db" exitCode=0 May 29 00:47:56.301642 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:56.301580 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" event={"ID":"f83593a1-25ed-4d6a-94f6-05e427aea6eb","Type":"ContainerDied","Data":"d18793b38fd393bd49a4d9bc1d7624dbdf61353e715ef4d9a19484afcb60d6db"} May 29 00:47:57.306562 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:57.306526 2551 generic.go:358] "Generic (PLEG): container finished" podID="f83593a1-25ed-4d6a-94f6-05e427aea6eb" containerID="37ca5d932220f283eb64280a7368440910b5ef669603815ce34ed140ae9d8c40" exitCode=0 May 29 00:47:57.306562 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:57.306565 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" event={"ID":"f83593a1-25ed-4d6a-94f6-05e427aea6eb","Type":"ContainerDied","Data":"37ca5d932220f283eb64280a7368440910b5ef669603815ce34ed140ae9d8c40"} May 29 00:47:58.214769 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:58.214586 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:47:58.214952 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:58.214799 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:47:58.214952 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:58.214725 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 00:47:58.214952 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:58.214891 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls podName:a0aa4161-0aa4-4e61-b55e-d737c3bfaecd nodeName:}" failed. No retries permitted until 2026-05-29 00:48:02.214874819 +0000 UTC m=+40.675078247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls") pod "dns-default-n5lpq" (UID: "a0aa4161-0aa4-4e61-b55e-d737c3bfaecd") : secret "dns-default-metrics-tls" not found May 29 00:47:58.214952 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:58.214924 2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 00:47:58.215116 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:47:58.214975 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert podName:bec90e87-7a28-4f14-83c6-61def84dbed9 nodeName:}" failed. No retries permitted until 2026-05-29 00:48:02.214959752 +0000 UTC m=+40.675163179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert") pod "ingress-canary-n5szr" (UID: "bec90e87-7a28-4f14-83c6-61def84dbed9") : secret "canary-serving-cert" not found May 29 00:47:58.311109 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:58.311082 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" event={"ID":"f83593a1-25ed-4d6a-94f6-05e427aea6eb","Type":"ContainerStarted","Data":"d8b425d63a349663b29179a578c4e0fa4fae0ade0510c4a6245a3aabd8899484"} May 29 00:47:58.332870 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:47:58.332829 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4pr7f" podStartSLOduration=4.388073531 podStartE2EDuration="36.332816617s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:47:23.398670292 +0000 UTC m=+1.858873733" lastFinishedPulling="2026-05-29 00:47:55.343413378 +0000 UTC m=+33.803616819" observedRunningTime="2026-05-29 00:47:58.33151382 +0000 UTC m=+36.791717269" watchObservedRunningTime="2026-05-29 00:47:58.332816617 +0000 UTC m=+36.793020059" May 29 00:48:02.238973 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:02.238938 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:48:02.238973 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:02.238979 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:48:02.239383 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:02.239080 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 00:48:02.239383 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:02.239097 2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 00:48:02.239383 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:02.239140 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls podName:a0aa4161-0aa4-4e61-b55e-d737c3bfaecd nodeName:}" failed. No retries permitted until 2026-05-29 00:48:10.239124616 +0000 UTC m=+48.699328043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls") pod "dns-default-n5lpq" (UID: "a0aa4161-0aa4-4e61-b55e-d737c3bfaecd") : secret "dns-default-metrics-tls" not found May 29 00:48:02.239383 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:02.239153 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert podName:bec90e87-7a28-4f14-83c6-61def84dbed9 nodeName:}" failed. No retries permitted until 2026-05-29 00:48:10.239147626 +0000 UTC m=+48.699351054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert") pod "ingress-canary-n5szr" (UID: "bec90e87-7a28-4f14-83c6-61def84dbed9") : secret "canary-serving-cert" not found May 29 00:48:10.291426 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:10.291386 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:48:10.291426 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:10.291430 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:48:10.291973 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:10.291534 2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 00:48:10.291973 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:10.291591 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert podName:bec90e87-7a28-4f14-83c6-61def84dbed9 nodeName:}" failed. No retries permitted until 2026-05-29 00:48:26.291577113 +0000 UTC m=+64.751780540 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert") pod "ingress-canary-n5szr" (UID: "bec90e87-7a28-4f14-83c6-61def84dbed9") : secret "canary-serving-cert" not found May 29 00:48:10.291973 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:10.291534 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 00:48:10.291973 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:10.291660 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls podName:a0aa4161-0aa4-4e61-b55e-d737c3bfaecd nodeName:}" failed. No retries permitted until 2026-05-29 00:48:26.291648057 +0000 UTC m=+64.751851497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls") pod "dns-default-n5lpq" (UID: "a0aa4161-0aa4-4e61-b55e-d737c3bfaecd") : secret "dns-default-metrics-tls" not found May 29 00:48:18.140151 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.140121 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n"] May 29 00:48:18.182614 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.182585 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n"] May 29 00:48:18.182773 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.182645 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" May 29 00:48:18.187155 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.187129 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" May 29 00:48:18.187271 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.187232 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4hbrw\"" May 29 00:48:18.187338 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.187315 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" May 29 00:48:18.187393 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.187374 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" May 29 00:48:18.187492 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.187476 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" May 29 00:48:18.343116 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.343089 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14667b39-73c4-45e6-a9e1-d7fb935361e2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8676556566-wsm6n\" (UID: \"14667b39-73c4-45e6-a9e1-d7fb935361e2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" May 29 00:48:18.343274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.343145 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf2ph\" (UniqueName: \"kubernetes.io/projected/14667b39-73c4-45e6-a9e1-d7fb935361e2-kube-api-access-pf2ph\") pod \"managed-serviceaccount-addon-agent-8676556566-wsm6n\" (UID: \"14667b39-73c4-45e6-a9e1-d7fb935361e2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" May 29 00:48:18.443767 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.443690 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pf2ph\" (UniqueName: \"kubernetes.io/projected/14667b39-73c4-45e6-a9e1-d7fb935361e2-kube-api-access-pf2ph\") pod \"managed-serviceaccount-addon-agent-8676556566-wsm6n\" (UID: \"14667b39-73c4-45e6-a9e1-d7fb935361e2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" May 29 00:48:18.443895 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.443875 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14667b39-73c4-45e6-a9e1-d7fb935361e2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8676556566-wsm6n\" (UID: \"14667b39-73c4-45e6-a9e1-d7fb935361e2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" May 29 00:48:18.446852 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.446831 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14667b39-73c4-45e6-a9e1-d7fb935361e2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8676556566-wsm6n\" (UID: \"14667b39-73c4-45e6-a9e1-d7fb935361e2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" May 29 00:48:18.452215 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.452197 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf2ph\" (UniqueName: \"kubernetes.io/projected/14667b39-73c4-45e6-a9e1-d7fb935361e2-kube-api-access-pf2ph\") pod \"managed-serviceaccount-addon-agent-8676556566-wsm6n\" (UID: \"14667b39-73c4-45e6-a9e1-d7fb935361e2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" May 29 00:48:18.503857 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.503837 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" May 29 00:48:18.667968 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:18.667941 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n"] May 29 00:48:18.670994 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:48:18.670972 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14667b39_73c4_45e6_a9e1_d7fb935361e2.slice/crio-146ef0a05a3155d04f6c25771b576dad6e5104c6ab873a040394290c5195cc33 WatchSource:0}: Error finding container 146ef0a05a3155d04f6c25771b576dad6e5104c6ab873a040394290c5195cc33: Status 404 returned error can't find the container with id 146ef0a05a3155d04f6c25771b576dad6e5104c6ab873a040394290c5195cc33 May 29 00:48:19.349505 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:19.349471 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" event={"ID":"14667b39-73c4-45e6-a9e1-d7fb935361e2","Type":"ContainerStarted","Data":"146ef0a05a3155d04f6c25771b576dad6e5104c6ab873a040394290c5195cc33"} May 29 00:48:20.299926 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:20.299900 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4vn9b" May 29 00:48:21.356542 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:21.355888 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" event={"ID":"14667b39-73c4-45e6-a9e1-d7fb935361e2","Type":"ContainerStarted","Data":"f7352e9a992b7ecbf887d3ef4bb34cb105852afda025ed18543fd3f0d11abae0"} May 29 00:48:21.375681 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:21.375635 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" podStartSLOduration=0.982318032 podStartE2EDuration="3.375622911s" podCreationTimestamp="2026-05-29 00:48:18 +0000 UTC" firstStartedPulling="2026-05-29 00:48:18.673242293 +0000 UTC m=+57.133445721" lastFinishedPulling="2026-05-29 00:48:21.066547172 +0000 UTC m=+59.526750600" observedRunningTime="2026-05-29 00:48:21.374786095 +0000 UTC m=+59.834989539" watchObservedRunningTime="2026-05-29 00:48:21.375622911 +0000 UTC m=+59.835826360" May 29 00:48:26.296139 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:26.296104 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:48:26.296139 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:26.296144 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:48:26.296563 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:26.296243 2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 00:48:26.296563 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:26.296271 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 00:48:26.296563 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:26.296305 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert podName:bec90e87-7a28-4f14-83c6-61def84dbed9 nodeName:}" failed. No retries permitted until 2026-05-29 00:48:58.296291854 +0000 UTC m=+96.756495282 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert") pod "ingress-canary-n5szr" (UID: "bec90e87-7a28-4f14-83c6-61def84dbed9") : secret "canary-serving-cert" not found May 29 00:48:26.296563 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:26.296340 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls podName:a0aa4161-0aa4-4e61-b55e-d737c3bfaecd nodeName:}" failed. No retries permitted until 2026-05-29 00:48:58.296320509 +0000 UTC m=+96.756523936 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls") pod "dns-default-n5lpq" (UID: "a0aa4161-0aa4-4e61-b55e-d737c3bfaecd") : secret "dns-default-metrics-tls" not found May 29 00:48:26.800730 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:26.800701 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:48:26.803876 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:26.803858 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 29 00:48:26.811018 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:26.810999 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 29 00:48:26.811079 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:26.811053 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs podName:12335c23-f1a8-4eae-9210-8801fa6ce0bd nodeName:}" failed. No retries permitted until 2026-05-29 00:49:30.811037503 +0000 UTC m=+129.271240935 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs") pod "network-metrics-daemon-px5qc" (UID: "12335c23-f1a8-4eae-9210-8801fa6ce0bd") : secret "metrics-daemon-secret" not found May 29 00:48:26.901671 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:26.901645 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:48:26.904356 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:26.904339 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 29 00:48:26.914878 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:26.914861 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 29 00:48:26.925226 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:26.925202 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slkrw\" (UniqueName: \"kubernetes.io/projected/75386284-5f2d-431c-8d8d-19a580679c5b-kube-api-access-slkrw\") pod \"network-check-target-6fjlq\" (UID: \"75386284-5f2d-431c-8d8d-19a580679c5b\") " pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:48:27.049519 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:27.049492 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c2thf\"" May 29 00:48:27.056985 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:27.056940 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:48:27.171436 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:27.171370 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6fjlq"] May 29 00:48:27.173989 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:48:27.173966 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75386284_5f2d_431c_8d8d_19a580679c5b.slice/crio-7cc84dca6cd15468c8041c8b5a1aea1e5d9cb9a53efc32dbaa40d5059a70d2ff WatchSource:0}: Error finding container 7cc84dca6cd15468c8041c8b5a1aea1e5d9cb9a53efc32dbaa40d5059a70d2ff: Status 404 returned error can't find the container with id 7cc84dca6cd15468c8041c8b5a1aea1e5d9cb9a53efc32dbaa40d5059a70d2ff May 29 00:48:27.366459 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:27.366398 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6fjlq" event={"ID":"75386284-5f2d-431c-8d8d-19a580679c5b","Type":"ContainerStarted","Data":"7cc84dca6cd15468c8041c8b5a1aea1e5d9cb9a53efc32dbaa40d5059a70d2ff"} May 29 00:48:30.374235 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:30.374191 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6fjlq" event={"ID":"75386284-5f2d-431c-8d8d-19a580679c5b","Type":"ContainerStarted","Data":"bf82e75632d66213df477c8b44e053097f53baaf81d7638ff2f912e7872fdf98"} May 29 00:48:30.374581 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:30.374333 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:48:30.390111 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:30.390072 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6fjlq" podStartSLOduration=65.83603464 podStartE2EDuration="1m8.390062047s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:48:27.176316381 +0000 UTC m=+65.636519809" lastFinishedPulling="2026-05-29 00:48:29.730343783 +0000 UTC m=+68.190547216" observedRunningTime="2026-05-29 00:48:30.389260526 +0000 UTC m=+68.849463976" watchObservedRunningTime="2026-05-29 00:48:30.390062047 +0000 UTC m=+68.850265488" May 29 00:48:49.779659 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.779549 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5fc48bf698-5bp8x"] May 29 00:48:49.783928 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.783910 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.786484 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.786463 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 29 00:48:49.786746 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.786730 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w7cln\"" May 29 00:48:49.786746 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.786739 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 29 00:48:49.786893 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.786880 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 29 00:48:49.786949 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.786920 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 29 00:48:49.787125 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.787107 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 29 00:48:49.787845 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.787828 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 29 00:48:49.794969 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.794950 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5fc48bf698-5bp8x"] May 29 00:48:49.845299 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.845275 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.845386 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.845316 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-default-certificate\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.845386 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.845343 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jdwk\" (UniqueName: \"kubernetes.io/projected/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-kube-api-access-2jdwk\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.845469 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.845446 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.845511 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.845473 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-stats-auth\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.882448 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.882426 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg"] May 29 00:48:49.885084 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.885069 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg" May 29 00:48:49.887882 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.887867 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-p8kp6\"" May 29 00:48:49.888140 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.888126 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" May 29 00:48:49.889274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.889257 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" May 29 00:48:49.896868 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.896849 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg"] May 29 00:48:49.946115 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.946090 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-stats-auth\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.946269 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.946128 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.946269 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.946158 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7llf\" (UniqueName: \"kubernetes.io/projected/9cb49a07-dae6-457b-bd3b-a054879e505c-kube-api-access-g7llf\") pod \"volume-data-source-validator-5cf8c89d64-rhqlg\" (UID: \"9cb49a07-dae6-457b-bd3b-a054879e505c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg" May 29 00:48:49.946269 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.946200 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-default-certificate\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.946269 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.946229 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2jdwk\" (UniqueName: \"kubernetes.io/projected/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-kube-api-access-2jdwk\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.946465 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:49.946277 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:48:50.44626071 +0000 UTC m=+88.906464138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : configmap references non-existent config key: service-ca.crt May 29 00:48:49.946465 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.946436 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.946596 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:49.946570 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 29 00:48:49.946669 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:49.946632 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:48:50.446619558 +0000 UTC m=+88.906822986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : secret "router-metrics-certs-default" not found May 29 00:48:49.948530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.948508 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-stats-auth\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.948625 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.948542 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-default-certificate\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:49.956898 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:49.956878 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jdwk\" (UniqueName: \"kubernetes.io/projected/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-kube-api-access-2jdwk\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:50.047291 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.047214 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7llf\" (UniqueName: \"kubernetes.io/projected/9cb49a07-dae6-457b-bd3b-a054879e505c-kube-api-access-g7llf\") pod \"volume-data-source-validator-5cf8c89d64-rhqlg\" (UID: \"9cb49a07-dae6-457b-bd3b-a054879e505c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg" May 29 00:48:50.054569 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.054546 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7llf\" (UniqueName: \"kubernetes.io/projected/9cb49a07-dae6-457b-bd3b-a054879e505c-kube-api-access-g7llf\") pod \"volume-data-source-validator-5cf8c89d64-rhqlg\" (UID: \"9cb49a07-dae6-457b-bd3b-a054879e505c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg" May 29 00:48:50.192616 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.192586 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg" May 29 00:48:50.303875 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.303816 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg"] May 29 00:48:50.307155 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:48:50.307124 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb49a07_dae6_457b_bd3b_a054879e505c.slice/crio-2818ebdf94e8616cf355a00eeee2b5f08fd0e1c929c22ee6731b6b3e2e9f22db WatchSource:0}: Error finding container 2818ebdf94e8616cf355a00eeee2b5f08fd0e1c929c22ee6731b6b3e2e9f22db: Status 404 returned error can't find the container with id 2818ebdf94e8616cf355a00eeee2b5f08fd0e1c929c22ee6731b6b3e2e9f22db May 29 00:48:50.411120 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.411092 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg" event={"ID":"9cb49a07-dae6-457b-bd3b-a054879e505c","Type":"ContainerStarted","Data":"2818ebdf94e8616cf355a00eeee2b5f08fd0e1c929c22ee6731b6b3e2e9f22db"} May 29 00:48:50.450596 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.450568 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:50.450693 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.450614 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:50.450774 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:50.450719 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:48:51.450704922 +0000 UTC m=+89.910908350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : configmap references non-existent config key: service-ca.crt May 29 00:48:50.450774 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:50.450719 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 29 00:48:50.450870 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:50.450800 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:48:51.450780332 +0000 UTC m=+89.910983773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : secret "router-metrics-certs-default" not found May 29 00:48:50.710321 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.710266 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-m6z2w"] May 29 00:48:50.714290 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.714274 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.716722 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.716688 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" May 29 00:48:50.717035 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.717011 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" May 29 00:48:50.717118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.717065 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" May 29 00:48:50.718306 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.718287 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" May 29 00:48:50.718396 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.718315 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-thgc7\"" May 29 00:48:50.723591 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.723570 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" May 29 00:48:50.723697 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.723633 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-m6z2w"] May 29 00:48:50.753415 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.753393 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-config\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.753498 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.753418 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-trusted-ca\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.753544 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.753529 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9gft\" (UniqueName: \"kubernetes.io/projected/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-kube-api-access-w9gft\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.753580 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.753562 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-serving-cert\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.854351 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.854326 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-serving-cert\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.854720 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.854366 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-config\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.854720 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.854391 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-trusted-ca\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.854720 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.854513 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9gft\" (UniqueName: \"kubernetes.io/projected/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-kube-api-access-w9gft\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.855236 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.855212 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-config\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.855904 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.855886 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-trusted-ca\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.856919 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.856898 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-serving-cert\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:50.863416 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:50.863394 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9gft\" (UniqueName: \"kubernetes.io/projected/bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2-kube-api-access-w9gft\") pod \"console-operator-76dff48d59-m6z2w\" (UID: \"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2\") " pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:51.025089 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:51.025063 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:48:51.157115 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:51.157086 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-m6z2w"] May 29 00:48:51.160521 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:48:51.160488 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdeff395_20f1_4a4e_8e7a_f0bc2f5feac2.slice/crio-fb517ac340de63a8d153cced0f6ab73b65151d880696493b52d8ab4abfeb99af WatchSource:0}: Error finding container fb517ac340de63a8d153cced0f6ab73b65151d880696493b52d8ab4abfeb99af: Status 404 returned error can't find the container with id fb517ac340de63a8d153cced0f6ab73b65151d880696493b52d8ab4abfeb99af May 29 00:48:51.414180 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:51.414117 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" event={"ID":"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2","Type":"ContainerStarted","Data":"fb517ac340de63a8d153cced0f6ab73b65151d880696493b52d8ab4abfeb99af"} May 29 00:48:51.459511 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:51.459488 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:51.459613 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:51.459528 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:51.459658 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:51.459624 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 29 00:48:51.459658 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:51.459637 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:48:53.459623266 +0000 UTC m=+91.919826694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : configmap references non-existent config key: service-ca.crt May 29 00:48:51.459736 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:51.459671 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:48:53.459656643 +0000 UTC m=+91.919860070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : secret "router-metrics-certs-default" not found May 29 00:48:52.417709 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.417671 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg" event={"ID":"9cb49a07-dae6-457b-bd3b-a054879e505c","Type":"ContainerStarted","Data":"64c0fb673d8a9f142a027eccd1b02cf5e17658f3ad860fc6712ef68271ff48b1"} May 29 00:48:52.432484 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.432424 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-rhqlg" podStartSLOduration=1.796118974 podStartE2EDuration="3.432408382s" podCreationTimestamp="2026-05-29 00:48:49 +0000 UTC" firstStartedPulling="2026-05-29 00:48:50.308811834 +0000 UTC m=+88.769015262" lastFinishedPulling="2026-05-29 00:48:51.945101236 +0000 UTC m=+90.405304670" observedRunningTime="2026-05-29 00:48:52.431934577 +0000 UTC m=+90.892138028" watchObservedRunningTime="2026-05-29 00:48:52.432408382 +0000 UTC m=+90.892611838" May 29 00:48:52.817418 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.817381 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7ff5466cfd-2l8kf"] May 29 00:48:52.820516 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.820501 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.823174 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.823149 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnqb5\"" May 29 00:48:52.823174 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.823164 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 29 00:48:52.823337 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.823151 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 29 00:48:52.823337 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.823273 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 29 00:48:52.828381 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.828360 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 29 00:48:52.832007 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.831971 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7ff5466cfd-2l8kf"] May 29 00:48:52.870972 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.870950 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-trusted-ca\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.871085 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.870989 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-installation-pull-secrets\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.871085 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.871034 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-bound-sa-token\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.871085 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.871070 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-certificates\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.871263 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.871103 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-image-registry-private-configuration\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.871263 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.871202 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.871263 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.871233 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/effe3781-f0b5-46fc-b17c-f57925d0dffd-ca-trust-extracted\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.871409 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.871268 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzhmk\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-kube-api-access-pzhmk\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972138 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972102 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pzhmk\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-kube-api-access-pzhmk\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972303 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972163 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-trusted-ca\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972303 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972202 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-installation-pull-secrets\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972303 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972263 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-bound-sa-token\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972472 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972315 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-certificates\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972472 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972352 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-image-registry-private-configuration\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972472 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972406 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972472 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972440 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/effe3781-f0b5-46fc-b17c-f57925d0dffd-ca-trust-extracted\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.972775 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:52.972739 2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 00:48:52.972857 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:52.972778 2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff5466cfd-2l8kf: secret "image-registry-tls" not found May 29 00:48:52.972857 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:52.972854 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls podName:effe3781-f0b5-46fc-b17c-f57925d0dffd nodeName:}" failed. No retries permitted until 2026-05-29 00:48:53.472829857 +0000 UTC m=+91.933033293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls") pod "image-registry-7ff5466cfd-2l8kf" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd") : secret "image-registry-tls" not found May 29 00:48:52.972984 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.972879 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/effe3781-f0b5-46fc-b17c-f57925d0dffd-ca-trust-extracted\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.973349 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.973326 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-certificates\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.973349 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.973348 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-trusted-ca\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.974911 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.974891 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-image-registry-private-configuration\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.975091 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.975069 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-installation-pull-secrets\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.982831 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.982809 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-bound-sa-token\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:52.983033 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:52.983012 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzhmk\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-kube-api-access-pzhmk\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:53.476186 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:53.476165 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:53.476567 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:53.476205 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:53.476567 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:53.476234 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:53.476567 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:53.476305 2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 00:48:53.476567 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:53.476325 2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff5466cfd-2l8kf: secret "image-registry-tls" not found May 29 00:48:53.476567 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:53.476379 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:48:57.476365126 +0000 UTC m=+95.936568553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : configmap references non-existent config key: service-ca.crt May 29 00:48:53.476567 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:53.476304 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 29 00:48:53.476567 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:53.476403 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls podName:effe3781-f0b5-46fc-b17c-f57925d0dffd nodeName:}" failed. No retries permitted until 2026-05-29 00:48:54.476392828 +0000 UTC m=+92.936596261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls") pod "image-registry-7ff5466cfd-2l8kf" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd") : secret "image-registry-tls" not found May 29 00:48:53.476567 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:53.476456 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:48:57.47643841 +0000 UTC m=+95.936641851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : secret "router-metrics-certs-default" not found May 29 00:48:54.422331 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:54.422301 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/0.log" May 29 00:48:54.422481 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:54.422340 2551 generic.go:358] "Generic (PLEG): container finished" podID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" containerID="051d31dee9686a35a184b633eeb02302f22690868e02fdf27ec22e42f0f17cd8" exitCode=255 May 29 00:48:54.422481 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:54.422364 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" event={"ID":"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2","Type":"ContainerDied","Data":"051d31dee9686a35a184b633eeb02302f22690868e02fdf27ec22e42f0f17cd8"} May 29 00:48:54.422639 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:54.422624 2551 scope.go:117] "RemoveContainer" containerID="051d31dee9686a35a184b633eeb02302f22690868e02fdf27ec22e42f0f17cd8" May 29 00:48:54.484016 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:54.483990 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:54.484400 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:54.484177 2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 00:48:54.484400 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:54.484196 2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff5466cfd-2l8kf: secret "image-registry-tls" not found May 29 00:48:54.484400 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:54.484252 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls podName:effe3781-f0b5-46fc-b17c-f57925d0dffd nodeName:}" failed. No retries permitted until 2026-05-29 00:48:56.484232869 +0000 UTC m=+94.944436310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls") pod "image-registry-7ff5466cfd-2l8kf" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd") : secret "image-registry-tls" not found May 29 00:48:55.171632 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:55.171603 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4tt5k_31951da8-a753-4b89-9838-3f0c2c5505d1/dns-node-resolver/0.log" May 29 00:48:55.425873 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:55.425807 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/1.log" May 29 00:48:55.426173 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:55.426158 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/0.log" May 29 00:48:55.426224 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:55.426192 2551 generic.go:358] "Generic (PLEG): container finished" podID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" containerID="ff6b74941cbdfa6d693a2311a366d6b3779f075f4b47e9be69f0f0e62ff35a20" exitCode=255 May 29 00:48:55.426280 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:55.426250 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" event={"ID":"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2","Type":"ContainerDied","Data":"ff6b74941cbdfa6d693a2311a366d6b3779f075f4b47e9be69f0f0e62ff35a20"} May 29 00:48:55.426280 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:55.426277 2551 scope.go:117] "RemoveContainer" containerID="051d31dee9686a35a184b633eeb02302f22690868e02fdf27ec22e42f0f17cd8" May 29 00:48:55.426480 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:55.426461 2551 scope.go:117] "RemoveContainer" containerID="ff6b74941cbdfa6d693a2311a366d6b3779f075f4b47e9be69f0f0e62ff35a20" May 29 00:48:55.426650 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:55.426631 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-m6z2w_openshift-console-operator(bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2)\"" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" podUID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" May 29 00:48:56.171128 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:56.171104 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2zcx5_f0790f93-cef8-451c-8dc7-358ce10becf4/node-ca/0.log" May 29 00:48:56.429588 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:56.429520 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/1.log" May 29 00:48:56.429879 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:56.429861 2551 scope.go:117] "RemoveContainer" containerID="ff6b74941cbdfa6d693a2311a366d6b3779f075f4b47e9be69f0f0e62ff35a20" May 29 00:48:56.430051 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:56.430035 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-m6z2w_openshift-console-operator(bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2)\"" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" podUID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" May 29 00:48:56.500233 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:56.500204 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:48:56.500384 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:56.500306 2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 00:48:56.500384 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:56.500322 2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff5466cfd-2l8kf: secret "image-registry-tls" not found May 29 00:48:56.500513 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:56.500389 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls podName:effe3781-f0b5-46fc-b17c-f57925d0dffd nodeName:}" failed. No retries permitted until 2026-05-29 00:49:00.500361085 +0000 UTC m=+98.960564514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls") pod "image-registry-7ff5466cfd-2l8kf" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd") : secret "image-registry-tls" not found May 29 00:48:57.506785 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:57.506738 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:57.507137 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:57.506800 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:48:57.507137 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:57.506874 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 29 00:48:57.507137 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:57.506890 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:49:05.506877058 +0000 UTC m=+103.967080487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : configmap references non-existent config key: service-ca.crt May 29 00:48:57.507137 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:57.506925 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:49:05.506909725 +0000 UTC m=+103.967113153 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : secret "router-metrics-certs-default" not found May 29 00:48:58.312320 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.312292 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:48:58.312453 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.312336 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:48:58.312453 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:58.312420 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 29 00:48:58.312453 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:58.312437 2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 29 00:48:58.312548 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:58.312481 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls podName:a0aa4161-0aa4-4e61-b55e-d737c3bfaecd nodeName:}" failed. No retries permitted until 2026-05-29 00:50:02.312464127 +0000 UTC m=+160.772667555 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls") pod "dns-default-n5lpq" (UID: "a0aa4161-0aa4-4e61-b55e-d737c3bfaecd") : secret "dns-default-metrics-tls" not found May 29 00:48:58.312548 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:48:58.312495 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert podName:bec90e87-7a28-4f14-83c6-61def84dbed9 nodeName:}" failed. No retries permitted until 2026-05-29 00:50:02.312488429 +0000 UTC m=+160.772691856 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert") pod "ingress-canary-n5szr" (UID: "bec90e87-7a28-4f14-83c6-61def84dbed9") : secret "canary-serving-cert" not found May 29 00:48:58.493872 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.493846 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9"] May 29 00:48:58.497386 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.497371 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" May 29 00:48:58.499949 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.499920 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" May 29 00:48:58.499949 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.499945 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" May 29 00:48:58.500100 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.500021 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-482q7\"" May 29 00:48:58.504221 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.504198 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9"] May 29 00:48:58.615177 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.615089 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l47j9\" (UniqueName: \"kubernetes.io/projected/972f368e-2614-4c0e-8bae-5c008540add4-kube-api-access-l47j9\") pod \"migrator-b985f5d89-w99v9\" (UID: \"972f368e-2614-4c0e-8bae-5c008540add4\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" May 29 00:48:58.716528 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.716490 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l47j9\" (UniqueName: \"kubernetes.io/projected/972f368e-2614-4c0e-8bae-5c008540add4-kube-api-access-l47j9\") pod \"migrator-b985f5d89-w99v9\" (UID: \"972f368e-2614-4c0e-8bae-5c008540add4\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" May 29 00:48:58.724022 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.723998 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l47j9\" (UniqueName: \"kubernetes.io/projected/972f368e-2614-4c0e-8bae-5c008540add4-kube-api-access-l47j9\") pod \"migrator-b985f5d89-w99v9\" (UID: \"972f368e-2614-4c0e-8bae-5c008540add4\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" May 29 00:48:58.806054 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.806030 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" May 29 00:48:58.923172 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:58.923145 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9"] May 29 00:48:58.926194 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:48:58.926175 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972f368e_2614_4c0e_8bae_5c008540add4.slice/crio-3fdd08184da3cd4e2e5c2cb89370549e3dc0f81eff33fa5c55ea4d56f5c0e4dc WatchSource:0}: Error finding container 3fdd08184da3cd4e2e5c2cb89370549e3dc0f81eff33fa5c55ea4d56f5c0e4dc: Status 404 returned error can't find the container with id 3fdd08184da3cd4e2e5c2cb89370549e3dc0f81eff33fa5c55ea4d56f5c0e4dc May 29 00:48:59.436354 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:48:59.436315 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" event={"ID":"972f368e-2614-4c0e-8bae-5c008540add4","Type":"ContainerStarted","Data":"3fdd08184da3cd4e2e5c2cb89370549e3dc0f81eff33fa5c55ea4d56f5c0e4dc"} May 29 00:49:00.441170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:00.441099 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" event={"ID":"972f368e-2614-4c0e-8bae-5c008540add4","Type":"ContainerStarted","Data":"9dc8002a7e2c5fb4a03e3910f5126e0f4f85c8e95463a2db8b52fb14fd4d94cd"} May 29 00:49:00.441170 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:00.441132 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" event={"ID":"972f368e-2614-4c0e-8bae-5c008540add4","Type":"ContainerStarted","Data":"38531cf694a066770a5166d18b2af71e72ace6e88e729103df1752a0b14e2426"} May 29 00:49:00.456717 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:00.456678 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-w99v9" podStartSLOduration=1.309325957 podStartE2EDuration="2.456666287s" podCreationTimestamp="2026-05-29 00:48:58 +0000 UTC" firstStartedPulling="2026-05-29 00:48:58.928058554 +0000 UTC m=+97.388261981" lastFinishedPulling="2026-05-29 00:49:00.07539888 +0000 UTC m=+98.535602311" observedRunningTime="2026-05-29 00:49:00.454951679 +0000 UTC m=+98.915155142" watchObservedRunningTime="2026-05-29 00:49:00.456666287 +0000 UTC m=+98.916869737" May 29 00:49:00.529183 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:00.529162 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:49:00.529314 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:00.529295 2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 00:49:00.529358 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:00.529316 2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff5466cfd-2l8kf: secret "image-registry-tls" not found May 29 00:49:00.529391 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:00.529366 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls podName:effe3781-f0b5-46fc-b17c-f57925d0dffd nodeName:}" failed. No retries permitted until 2026-05-29 00:49:08.529350494 +0000 UTC m=+106.989553941 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls") pod "image-registry-7ff5466cfd-2l8kf" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd") : secret "image-registry-tls" not found May 29 00:49:01.025775 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:01.025733 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:49:01.025775 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:01.025784 2551 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:49:01.026073 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:01.026061 2551 scope.go:117] "RemoveContainer" containerID="ff6b74941cbdfa6d693a2311a366d6b3779f075f4b47e9be69f0f0e62ff35a20" May 29 00:49:01.026209 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:01.026194 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-m6z2w_openshift-console-operator(bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2)\"" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" podUID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" May 29 00:49:01.378824 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:01.378723 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6fjlq" May 29 00:49:05.565425 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:05.565393 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:05.565812 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:05.565445 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:05.565812 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:05.565527 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 29 00:49:05.565812 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:05.565589 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:49:21.565571998 +0000 UTC m=+120.025775446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : configmap references non-existent config key: service-ca.crt May 29 00:49:05.565812 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:05.565605 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs podName:8eaf00ba-f505-4f03-8dc0-599c0d80af9f nodeName:}" failed. No retries permitted until 2026-05-29 00:49:21.5655985 +0000 UTC m=+120.025801928 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs") pod "router-default-5fc48bf698-5bp8x" (UID: "8eaf00ba-f505-4f03-8dc0-599c0d80af9f") : secret "router-metrics-certs-default" not found May 29 00:49:08.590204 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:08.590169 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls\") pod \"image-registry-7ff5466cfd-2l8kf\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:49:08.590703 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:08.590277 2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 29 00:49:08.590703 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:08.590289 2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7ff5466cfd-2l8kf: secret "image-registry-tls" not found May 29 00:49:08.590703 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:08.590348 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls podName:effe3781-f0b5-46fc-b17c-f57925d0dffd nodeName:}" failed. No retries permitted until 2026-05-29 00:49:24.590333358 +0000 UTC m=+123.050536786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls") pod "image-registry-7ff5466cfd-2l8kf" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd") : secret "image-registry-tls" not found May 29 00:49:16.139568 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:16.139534 2551 scope.go:117] "RemoveContainer" containerID="ff6b74941cbdfa6d693a2311a366d6b3779f075f4b47e9be69f0f0e62ff35a20" May 29 00:49:16.481932 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:16.481905 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 00:49:16.482254 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:16.482237 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/1.log" May 29 00:49:16.482364 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:16.482283 2551 generic.go:358] "Generic (PLEG): container finished" podID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" containerID="29f6927340928a96a0f4cc2cf63c7d0b6d48050c4c200a29b412e32daaa091f9" exitCode=255 May 29 00:49:16.482364 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:16.482346 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" event={"ID":"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2","Type":"ContainerDied","Data":"29f6927340928a96a0f4cc2cf63c7d0b6d48050c4c200a29b412e32daaa091f9"} May 29 00:49:16.482472 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:16.482386 2551 scope.go:117] "RemoveContainer" containerID="ff6b74941cbdfa6d693a2311a366d6b3779f075f4b47e9be69f0f0e62ff35a20" May 29 00:49:16.482709 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:16.482685 2551 scope.go:117] "RemoveContainer" containerID="29f6927340928a96a0f4cc2cf63c7d0b6d48050c4c200a29b412e32daaa091f9" May 29 00:49:16.482890 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:16.482873 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-m6z2w_openshift-console-operator(bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2)\"" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" podUID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" May 29 00:49:17.486039 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:17.486011 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 00:49:21.025155 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.025122 2551 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:49:21.025708 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.025173 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:49:21.025708 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.025501 2551 scope.go:117] "RemoveContainer" containerID="29f6927340928a96a0f4cc2cf63c7d0b6d48050c4c200a29b412e32daaa091f9" May 29 00:49:21.025708 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:21.025697 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-m6z2w_openshift-console-operator(bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2)\"" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" podUID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" May 29 00:49:21.585473 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.585437 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:21.585646 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.585495 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:21.586036 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.586009 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-service-ca-bundle\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:21.587828 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.587808 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8eaf00ba-f505-4f03-8dc0-599c0d80af9f-metrics-certs\") pod \"router-default-5fc48bf698-5bp8x\" (UID: \"8eaf00ba-f505-4f03-8dc0-599c0d80af9f\") " pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:21.592610 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.592585 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:21.716558 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:21.716529 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5fc48bf698-5bp8x"] May 29 00:49:21.719581 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:21.719556 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eaf00ba_f505_4f03_8dc0_599c0d80af9f.slice/crio-008858bd00564191a6953aed53b8c229ef88a17500bf736c9469403b1bf08322 WatchSource:0}: Error finding container 008858bd00564191a6953aed53b8c229ef88a17500bf736c9469403b1bf08322: Status 404 returned error can't find the container with id 008858bd00564191a6953aed53b8c229ef88a17500bf736c9469403b1bf08322 May 29 00:49:22.499292 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.499254 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" event={"ID":"8eaf00ba-f505-4f03-8dc0-599c0d80af9f","Type":"ContainerStarted","Data":"b64246f6813cc371aea2486773ad793db9bff2290284b076a3c0f73024f5f210"} May 29 00:49:22.499292 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.499291 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" event={"ID":"8eaf00ba-f505-4f03-8dc0-599c0d80af9f","Type":"ContainerStarted","Data":"008858bd00564191a6953aed53b8c229ef88a17500bf736c9469403b1bf08322"} May 29 00:49:22.522224 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.522194 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rn29k"] May 29 00:49:22.523379 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.523339 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" podStartSLOduration=33.523329117 podStartE2EDuration="33.523329117s" podCreationTimestamp="2026-05-29 00:48:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 00:49:22.523034027 +0000 UTC m=+120.983237474" watchObservedRunningTime="2026-05-29 00:49:22.523329117 +0000 UTC m=+120.983532566" May 29 00:49:22.524346 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.524330 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.526946 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.526926 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 29 00:49:22.527096 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.527077 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 29 00:49:22.527161 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.527132 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 29 00:49:22.527161 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.527149 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 29 00:49:22.527304 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.527290 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4g569\"" May 29 00:49:22.535203 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.535183 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rn29k"] May 29 00:49:22.586864 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.586840 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7ff5466cfd-2l8kf"] May 29 00:49:22.586982 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:22.586966 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" podUID="effe3781-f0b5-46fc-b17c-f57925d0dffd" May 29 00:49:22.592110 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.592084 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdd6492a-262d-4198-8b94-a212d75eba2e-data-volume\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.592160 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.592120 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdd6492a-262d-4198-8b94-a212d75eba2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.592259 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.592241 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5g4b\" (UniqueName: \"kubernetes.io/projected/bdd6492a-262d-4198-8b94-a212d75eba2e-kube-api-access-j5g4b\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.592318 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.592271 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bdd6492a-262d-4198-8b94-a212d75eba2e-crio-socket\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.592374 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.592352 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdd6492a-262d-4198-8b94-a212d75eba2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.593054 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.593037 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:22.595300 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.595280 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:22.693590 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.693566 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j5g4b\" (UniqueName: \"kubernetes.io/projected/bdd6492a-262d-4198-8b94-a212d75eba2e-kube-api-access-j5g4b\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.693686 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.693599 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bdd6492a-262d-4198-8b94-a212d75eba2e-crio-socket\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.693686 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.693626 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdd6492a-262d-4198-8b94-a212d75eba2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.693836 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.693701 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bdd6492a-262d-4198-8b94-a212d75eba2e-crio-socket\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.693836 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.693787 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdd6492a-262d-4198-8b94-a212d75eba2e-data-volume\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.693836 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.693820 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdd6492a-262d-4198-8b94-a212d75eba2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.694068 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.694051 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdd6492a-262d-4198-8b94-a212d75eba2e-data-volume\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.694137 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.694092 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bdd6492a-262d-4198-8b94-a212d75eba2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.695965 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.695947 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bdd6492a-262d-4198-8b94-a212d75eba2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.705899 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.705877 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5g4b\" (UniqueName: \"kubernetes.io/projected/bdd6492a-262d-4198-8b94-a212d75eba2e-kube-api-access-j5g4b\") pod \"insights-runtime-extractor-rn29k\" (UID: \"bdd6492a-262d-4198-8b94-a212d75eba2e\") " pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.832365 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.832310 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rn29k" May 29 00:49:22.950562 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:22.950536 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rn29k"] May 29 00:49:22.953614 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:22.953588 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdd6492a_262d_4198_8b94_a212d75eba2e.slice/crio-8d4ad368721ebc3ac2fe7e7ae4590730635adf945bc4ab87de278025cbe2950a WatchSource:0}: Error finding container 8d4ad368721ebc3ac2fe7e7ae4590730635adf945bc4ab87de278025cbe2950a: Status 404 returned error can't find the container with id 8d4ad368721ebc3ac2fe7e7ae4590730635adf945bc4ab87de278025cbe2950a May 29 00:49:23.503325 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.503289 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rn29k" event={"ID":"bdd6492a-262d-4198-8b94-a212d75eba2e","Type":"ContainerStarted","Data":"0b89bf4b642aeca9f5e30f854b0d6f9bb4591093bb0e49ec915b088e532987a3"} May 29 00:49:23.503667 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.503332 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rn29k" event={"ID":"bdd6492a-262d-4198-8b94-a212d75eba2e","Type":"ContainerStarted","Data":"8d4ad368721ebc3ac2fe7e7ae4590730635adf945bc4ab87de278025cbe2950a"} May 29 00:49:23.503667 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.503303 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:49:23.503667 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.503623 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:23.504983 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.504961 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5fc48bf698-5bp8x" May 29 00:49:23.507591 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.507575 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:49:23.599892 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.599873 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/effe3781-f0b5-46fc-b17c-f57925d0dffd-ca-trust-extracted\") pod \"effe3781-f0b5-46fc-b17c-f57925d0dffd\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " May 29 00:49:23.599988 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.599915 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-bound-sa-token\") pod \"effe3781-f0b5-46fc-b17c-f57925d0dffd\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " May 29 00:49:23.599988 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.599939 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-image-registry-private-configuration\") pod \"effe3781-f0b5-46fc-b17c-f57925d0dffd\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " May 29 00:49:23.600061 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.599996 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-installation-pull-secrets\") pod \"effe3781-f0b5-46fc-b17c-f57925d0dffd\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " May 29 00:49:23.600061 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.600036 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzhmk\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-kube-api-access-pzhmk\") pod \"effe3781-f0b5-46fc-b17c-f57925d0dffd\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " May 29 00:49:23.600147 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.600073 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-trusted-ca\") pod \"effe3781-f0b5-46fc-b17c-f57925d0dffd\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " May 29 00:49:23.600147 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.600099 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-certificates\") pod \"effe3781-f0b5-46fc-b17c-f57925d0dffd\" (UID: \"effe3781-f0b5-46fc-b17c-f57925d0dffd\") " May 29 00:49:23.600147 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.600129 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effe3781-f0b5-46fc-b17c-f57925d0dffd-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "effe3781-f0b5-46fc-b17c-f57925d0dffd" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 29 00:49:23.600469 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.600442 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "effe3781-f0b5-46fc-b17c-f57925d0dffd" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:49:23.600564 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.600486 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "effe3781-f0b5-46fc-b17c-f57925d0dffd" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:49:23.601025 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.600958 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-trusted-ca\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:49:23.601025 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.600987 2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-certificates\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:49:23.601025 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.601003 2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/effe3781-f0b5-46fc-b17c-f57925d0dffd-ca-trust-extracted\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:49:23.602156 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.602132 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "effe3781-f0b5-46fc-b17c-f57925d0dffd" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 00:49:23.602237 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.602158 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "effe3781-f0b5-46fc-b17c-f57925d0dffd" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 00:49:23.602417 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.602390 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "effe3781-f0b5-46fc-b17c-f57925d0dffd" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 00:49:23.602417 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.602405 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-kube-api-access-pzhmk" (OuterVolumeSpecName: "kube-api-access-pzhmk") pod "effe3781-f0b5-46fc-b17c-f57925d0dffd" (UID: "effe3781-f0b5-46fc-b17c-f57925d0dffd"). InnerVolumeSpecName "kube-api-access-pzhmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 00:49:23.701986 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.701958 2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-bound-sa-token\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:49:23.701986 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.701985 2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-image-registry-private-configuration\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:49:23.702126 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.701996 2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/effe3781-f0b5-46fc-b17c-f57925d0dffd-installation-pull-secrets\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:49:23.702126 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:23.702007 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzhmk\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-kube-api-access-pzhmk\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:49:24.507997 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:24.507957 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rn29k" event={"ID":"bdd6492a-262d-4198-8b94-a212d75eba2e","Type":"ContainerStarted","Data":"0a8689e1f03fa0eaef71a2f668165be97d2809625b562ffe009df5e9bf2477ee"} May 29 00:49:24.508395 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:24.508109 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ff5466cfd-2l8kf" May 29 00:49:24.535967 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:24.535940 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7ff5466cfd-2l8kf"] May 29 00:49:24.541611 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:24.541590 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7ff5466cfd-2l8kf"] May 29 00:49:24.608982 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:24.608960 2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/effe3781-f0b5-46fc-b17c-f57925d0dffd-registry-tls\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:49:25.512278 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:25.512250 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rn29k" event={"ID":"bdd6492a-262d-4198-8b94-a212d75eba2e","Type":"ContainerStarted","Data":"b261d0756aadc36139e794b2d126fd34e8226b883bb7a59121650eef6d0ca089"} May 29 00:49:25.529850 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:25.529809 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rn29k" podStartSLOduration=1.335938469 podStartE2EDuration="3.529785927s" podCreationTimestamp="2026-05-29 00:49:22 +0000 UTC" firstStartedPulling="2026-05-29 00:49:23.007442113 +0000 UTC m=+121.467645542" lastFinishedPulling="2026-05-29 00:49:25.201289558 +0000 UTC m=+123.661493000" observedRunningTime="2026-05-29 00:49:25.528101191 +0000 UTC m=+123.988304640" watchObservedRunningTime="2026-05-29 00:49:25.529785927 +0000 UTC m=+123.989989377" May 29 00:49:26.140186 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:26.140157 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="effe3781-f0b5-46fc-b17c-f57925d0dffd" path="/var/lib/kubelet/pods/effe3781-f0b5-46fc-b17c-f57925d0dffd/volumes" May 29 00:49:29.114345 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.114306 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj"] May 29 00:49:29.117465 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.117443 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.123345 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.123308 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" May 29 00:49:29.123480 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.123347 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" May 29 00:49:29.123480 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.123386 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hllcz\"" May 29 00:49:29.123480 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.123448 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 29 00:49:29.123646 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.123489 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 29 00:49:29.123646 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.123532 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 29 00:49:29.127975 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.127947 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj"] May 29 00:49:29.242724 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.242693 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35f5edeb-8f82-4972-8a38-b39c237b60c4-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.242863 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.242778 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp76q\" (UniqueName: \"kubernetes.io/projected/35f5edeb-8f82-4972-8a38-b39c237b60c4-kube-api-access-lp76q\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.242934 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.242871 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35f5edeb-8f82-4972-8a38-b39c237b60c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.242934 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.242909 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35f5edeb-8f82-4972-8a38-b39c237b60c4-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.343397 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.343368 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lp76q\" (UniqueName: \"kubernetes.io/projected/35f5edeb-8f82-4972-8a38-b39c237b60c4-kube-api-access-lp76q\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.343521 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.343420 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35f5edeb-8f82-4972-8a38-b39c237b60c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.343521 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.343445 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35f5edeb-8f82-4972-8a38-b39c237b60c4-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.343521 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.343494 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35f5edeb-8f82-4972-8a38-b39c237b60c4-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.344177 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.344153 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35f5edeb-8f82-4972-8a38-b39c237b60c4-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.346110 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.346073 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35f5edeb-8f82-4972-8a38-b39c237b60c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.346213 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.346122 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35f5edeb-8f82-4972-8a38-b39c237b60c4-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.351142 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.351116 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp76q\" (UniqueName: \"kubernetes.io/projected/35f5edeb-8f82-4972-8a38-b39c237b60c4-kube-api-access-lp76q\") pod \"prometheus-operator-7779bbdd78-fmsxj\" (UID: \"35f5edeb-8f82-4972-8a38-b39c237b60c4\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.427898 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.427833 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" May 29 00:49:29.550949 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:29.550924 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj"] May 29 00:49:29.554582 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:29.554549 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f5edeb_8f82_4972_8a38_b39c237b60c4.slice/crio-4498d8c483ffbad9935f8329f37858c1166f0fba581845f665c9de19af2e3b78 WatchSource:0}: Error finding container 4498d8c483ffbad9935f8329f37858c1166f0fba581845f665c9de19af2e3b78: Status 404 returned error can't find the container with id 4498d8c483ffbad9935f8329f37858c1166f0fba581845f665c9de19af2e3b78 May 29 00:49:30.527143 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:30.527109 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" event={"ID":"35f5edeb-8f82-4972-8a38-b39c237b60c4","Type":"ContainerStarted","Data":"4498d8c483ffbad9935f8329f37858c1166f0fba581845f665c9de19af2e3b78"} May 29 00:49:30.853912 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:30.853836 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:49:30.856467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:30.856434 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12335c23-f1a8-4eae-9210-8801fa6ce0bd-metrics-certs\") pod \"network-metrics-daemon-px5qc\" (UID: \"12335c23-f1a8-4eae-9210-8801fa6ce0bd\") " pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:49:30.954600 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:30.954571 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rlrhs\"" May 29 00:49:30.961948 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:30.961931 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-px5qc" May 29 00:49:31.102527 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:31.102491 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-px5qc"] May 29 00:49:31.390661 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:31.390630 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12335c23_f1a8_4eae_9210_8801fa6ce0bd.slice/crio-796a39d50352be1f97b2f80ad9fa674b5ed57cc2a58a5c429915c9688e1f9e7e WatchSource:0}: Error finding container 796a39d50352be1f97b2f80ad9fa674b5ed57cc2a58a5c429915c9688e1f9e7e: Status 404 returned error can't find the container with id 796a39d50352be1f97b2f80ad9fa674b5ed57cc2a58a5c429915c9688e1f9e7e May 29 00:49:31.530424 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:31.530395 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-px5qc" event={"ID":"12335c23-f1a8-4eae-9210-8801fa6ce0bd","Type":"ContainerStarted","Data":"796a39d50352be1f97b2f80ad9fa674b5ed57cc2a58a5c429915c9688e1f9e7e"} May 29 00:49:31.532031 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:31.532010 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" event={"ID":"35f5edeb-8f82-4972-8a38-b39c237b60c4","Type":"ContainerStarted","Data":"12d82f340fcccf40f63efbe61fdeff374f56d6eb0078636ba316cce53244f3d2"} May 29 00:49:32.139785 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:32.139737 2551 scope.go:117] "RemoveContainer" containerID="29f6927340928a96a0f4cc2cf63c7d0b6d48050c4c200a29b412e32daaa091f9" May 29 00:49:32.139990 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:32.139969 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-m6z2w_openshift-console-operator(bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2)\"" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" podUID="bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2" May 29 00:49:32.536439 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:32.536401 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" event={"ID":"35f5edeb-8f82-4972-8a38-b39c237b60c4","Type":"ContainerStarted","Data":"e0d80c4394d1559216e49bcb208933a837425685629a032abc3a9b05945f591d"} May 29 00:49:32.552643 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:32.552597 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-fmsxj" podStartSLOduration=1.676241018 podStartE2EDuration="3.552580931s" podCreationTimestamp="2026-05-29 00:49:29 +0000 UTC" firstStartedPulling="2026-05-29 00:49:29.55625684 +0000 UTC m=+128.016460272" lastFinishedPulling="2026-05-29 00:49:31.432596753 +0000 UTC m=+129.892800185" observedRunningTime="2026-05-29 00:49:32.55204423 +0000 UTC m=+131.012247681" watchObservedRunningTime="2026-05-29 00:49:32.552580931 +0000 UTC m=+131.012784381" May 29 00:49:33.541093 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:33.541058 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-px5qc" event={"ID":"12335c23-f1a8-4eae-9210-8801fa6ce0bd","Type":"ContainerStarted","Data":"75da44f1de02d9bfa2940044ce8125fbdb844dc3058b64e7f80a927a6e95ed90"} May 29 00:49:33.541093 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:33.541097 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-px5qc" event={"ID":"12335c23-f1a8-4eae-9210-8801fa6ce0bd","Type":"ContainerStarted","Data":"9d16052eed22e065add01472669bb62e6bf79080779ef666fa04a917391283c3"} May 29 00:49:33.556720 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:33.556681 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-px5qc" podStartSLOduration=130.266304986 podStartE2EDuration="2m11.556666502s" podCreationTimestamp="2026-05-29 00:47:22 +0000 UTC" firstStartedPulling="2026-05-29 00:49:31.392448254 +0000 UTC m=+129.852651683" lastFinishedPulling="2026-05-29 00:49:32.682809767 +0000 UTC m=+131.143013199" observedRunningTime="2026-05-29 00:49:33.55588631 +0000 UTC m=+132.016089759" watchObservedRunningTime="2026-05-29 00:49:33.556666502 +0000 UTC m=+132.016869952" May 29 00:49:34.479404 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.479372 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tgcvp"] May 29 00:49:34.482385 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.482365 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.485595 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.485574 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 29 00:49:34.485719 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.485579 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 29 00:49:34.485858 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.485843 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ss4hn\"" May 29 00:49:34.485898 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.485869 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 29 00:49:34.585805 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.585784 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-textfile\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.586118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.585813 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.586118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.585832 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/764832ca-7360-4c3a-96a3-057d82c23654-metrics-client-ca\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.586118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.585856 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-wtmp\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.586118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.585918 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-tls\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.586118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.585996 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct9rs\" (UniqueName: \"kubernetes.io/projected/764832ca-7360-4c3a-96a3-057d82c23654-kube-api-access-ct9rs\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.586118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.586028 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-accelerators-collector-config\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.586118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.586052 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-sys\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.586118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.586078 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-root\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687040 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687020 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-textfile\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687164 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687047 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687164 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687068 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/764832ca-7360-4c3a-96a3-057d82c23654-metrics-client-ca\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687164 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687089 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-wtmp\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687164 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687116 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-tls\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687365 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687165 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ct9rs\" (UniqueName: \"kubernetes.io/projected/764832ca-7360-4c3a-96a3-057d82c23654-kube-api-access-ct9rs\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687365 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:34.687234 2551 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found May 29 00:49:34.687365 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687254 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-wtmp\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687365 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:34.687297 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-tls podName:764832ca-7360-4c3a-96a3-057d82c23654 nodeName:}" failed. No retries permitted until 2026-05-29 00:49:35.187277511 +0000 UTC m=+133.647480954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-tls") pod "node-exporter-tgcvp" (UID: "764832ca-7360-4c3a-96a3-057d82c23654") : secret "node-exporter-tls" not found May 29 00:49:34.687365 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687342 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-accelerators-collector-config\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687614 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687379 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-sys\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687614 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687385 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-textfile\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687614 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687436 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-sys\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687614 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687450 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-root\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687614 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687540 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/764832ca-7360-4c3a-96a3-057d82c23654-root\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687874 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687671 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/764832ca-7360-4c3a-96a3-057d82c23654-metrics-client-ca\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.687874 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.687840 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-accelerators-collector-config\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.689487 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.689467 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:34.695309 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:34.695288 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct9rs\" (UniqueName: \"kubernetes.io/projected/764832ca-7360-4c3a-96a3-057d82c23654-kube-api-access-ct9rs\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:35.189574 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:35.189543 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-tls\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:35.191780 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:35.191737 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/764832ca-7360-4c3a-96a3-057d82c23654-node-exporter-tls\") pod \"node-exporter-tgcvp\" (UID: \"764832ca-7360-4c3a-96a3-057d82c23654\") " pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:35.390616 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:35.390582 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tgcvp" May 29 00:49:35.399327 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:35.399298 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764832ca_7360_4c3a_96a3_057d82c23654.slice/crio-a675d21e8b715ad7eb13bace57a21db1ed42e30c94b4a3d9cfa7a4bf7b6cf341 WatchSource:0}: Error finding container a675d21e8b715ad7eb13bace57a21db1ed42e30c94b4a3d9cfa7a4bf7b6cf341: Status 404 returned error can't find the container with id a675d21e8b715ad7eb13bace57a21db1ed42e30c94b4a3d9cfa7a4bf7b6cf341 May 29 00:49:35.550898 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:35.550866 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tgcvp" event={"ID":"764832ca-7360-4c3a-96a3-057d82c23654","Type":"ContainerStarted","Data":"a675d21e8b715ad7eb13bace57a21db1ed42e30c94b4a3d9cfa7a4bf7b6cf341"} May 29 00:49:36.551919 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.551898 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-55d6f9fc57-khts4"] May 29 00:49:36.554461 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.554440 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.555254 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.555231 2551 generic.go:358] "Generic (PLEG): container finished" podID="764832ca-7360-4c3a-96a3-057d82c23654" containerID="b259e6da03488315c439be7a7c50b2e06ddac1b5d491d02c248605f83ec27116" exitCode=0 May 29 00:49:36.555329 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.555284 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tgcvp" event={"ID":"764832ca-7360-4c3a-96a3-057d82c23654","Type":"ContainerDied","Data":"b259e6da03488315c439be7a7c50b2e06ddac1b5d491d02c248605f83ec27116"} May 29 00:49:36.556836 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.556787 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" May 29 00:49:36.556836 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.556808 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1mbf3edctnnts\"" May 29 00:49:36.556972 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.556809 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" May 29 00:49:36.558070 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.558011 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" May 29 00:49:36.558070 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.558010 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2r9sw\"" May 29 00:49:36.558070 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.558051 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" May 29 00:49:36.558070 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.558043 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" May 29 00:49:36.565372 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.565352 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55d6f9fc57-khts4"] May 29 00:49:36.700267 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.700242 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.700398 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.700284 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krklg\" (UniqueName: \"kubernetes.io/projected/96d65807-0a2f-4ecc-a509-454fc349c6de-kube-api-access-krklg\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.700398 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.700367 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.700471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.700456 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.700573 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.700551 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-tls\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.700637 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.700605 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96d65807-0a2f-4ecc-a509-454fc349c6de-metrics-client-ca\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.700690 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.700647 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.700690 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.700684 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-grpc-tls\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.801229 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.801209 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.801326 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.801238 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-krklg\" (UniqueName: \"kubernetes.io/projected/96d65807-0a2f-4ecc-a509-454fc349c6de-kube-api-access-krklg\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.801326 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.801265 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.801430 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.801395 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.801475 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.801435 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-tls\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.801475 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.801462 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96d65807-0a2f-4ecc-a509-454fc349c6de-metrics-client-ca\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.801735 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.801596 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.801735 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.801654 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-grpc-tls\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.802259 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.802237 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96d65807-0a2f-4ecc-a509-454fc349c6de-metrics-client-ca\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.804027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.804003 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-tls\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.804124 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.804046 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-grpc-tls\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.804193 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.804144 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.804256 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.804201 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.804345 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.804326 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.804393 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.804370 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96d65807-0a2f-4ecc-a509-454fc349c6de-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.809483 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.809439 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krklg\" (UniqueName: \"kubernetes.io/projected/96d65807-0a2f-4ecc-a509-454fc349c6de-kube-api-access-krklg\") pod \"thanos-querier-55d6f9fc57-khts4\" (UID: \"96d65807-0a2f-4ecc-a509-454fc349c6de\") " pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.863106 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.863083 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:36.984638 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:36.984616 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55d6f9fc57-khts4"] May 29 00:49:36.986702 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:36.986675 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d65807_0a2f_4ecc_a509_454fc349c6de.slice/crio-b797a1dbede7b546fd6fb4fc9ddb84f4998fd36b602d7ae09d4ec6658c66722f WatchSource:0}: Error finding container b797a1dbede7b546fd6fb4fc9ddb84f4998fd36b602d7ae09d4ec6658c66722f: Status 404 returned error can't find the container with id b797a1dbede7b546fd6fb4fc9ddb84f4998fd36b602d7ae09d4ec6658c66722f May 29 00:49:37.558695 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:37.558653 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" event={"ID":"96d65807-0a2f-4ecc-a509-454fc349c6de","Type":"ContainerStarted","Data":"b797a1dbede7b546fd6fb4fc9ddb84f4998fd36b602d7ae09d4ec6658c66722f"} May 29 00:49:37.560374 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:37.560349 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tgcvp" event={"ID":"764832ca-7360-4c3a-96a3-057d82c23654","Type":"ContainerStarted","Data":"07c2bd7556fe44248b9bf2e77adb11ccae45e9a0fb93d52cf8e3b20b9a5e396d"} May 29 00:49:37.560374 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:37.560378 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tgcvp" event={"ID":"764832ca-7360-4c3a-96a3-057d82c23654","Type":"ContainerStarted","Data":"b9e78f67ab1e47208a07985ee0072a4792d411aad2e02ee1c7fe7e9a3ec6897d"} May 29 00:49:37.583889 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:37.583842 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tgcvp" podStartSLOduration=2.676420717 podStartE2EDuration="3.58383056s" podCreationTimestamp="2026-05-29 00:49:34 +0000 UTC" firstStartedPulling="2026-05-29 00:49:35.401404445 +0000 UTC m=+133.861607874" lastFinishedPulling="2026-05-29 00:49:36.308814289 +0000 UTC m=+134.769017717" observedRunningTime="2026-05-29 00:49:37.580643806 +0000 UTC m=+136.040847268" watchObservedRunningTime="2026-05-29 00:49:37.58383056 +0000 UTC m=+136.044034010" May 29 00:49:39.567295 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.567254 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" event={"ID":"96d65807-0a2f-4ecc-a509-454fc349c6de","Type":"ContainerStarted","Data":"2dfa4442ad00d8a40d80bd2f45d0ab93224979d0f87bd770e1651d6d61eb659f"} May 29 00:49:39.567624 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.567300 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" event={"ID":"96d65807-0a2f-4ecc-a509-454fc349c6de","Type":"ContainerStarted","Data":"cd127c3bc3757276c4fff2ce6cf9b3c17ea950dce8a327a3be78237133f6f5f2"} May 29 00:49:39.567624 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.567311 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" event={"ID":"96d65807-0a2f-4ecc-a509-454fc349c6de","Type":"ContainerStarted","Data":"82c0bbce5d5c3d1267ba5afc38951e92303d9169111bd4e3b31968453a5bf7b9"} May 29 00:49:39.683851 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.683822 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5cb7d9d955-fw67s"] May 29 00:49:39.686987 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.686972 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.689557 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.689533 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" May 29 00:49:39.689805 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.689728 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" May 29 00:49:39.689805 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.689774 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" May 29 00:49:39.689805 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.689789 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" May 29 00:49:39.690008 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.689792 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8mlkr\"" May 29 00:49:39.690008 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.689774 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" May 29 00:49:39.695513 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.695319 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" May 29 00:49:39.696329 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.696308 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cb7d9d955-fw67s"] May 29 00:49:39.827635 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.827582 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.827635 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.827621 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-telemeter-client-tls\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.827783 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.827642 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-serving-certs-ca-bundle\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.827783 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.827658 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.827783 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.827710 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg264\" (UniqueName: \"kubernetes.io/projected/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-kube-api-access-dg264\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.827783 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.827745 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-metrics-client-ca\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.827907 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.827802 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-federate-client-tls\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.827907 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.827829 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-secret-telemeter-client\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.928385 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.928361 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.928458 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.928398 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-telemeter-client-tls\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.928458 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.928419 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-serving-certs-ca-bundle\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.928537 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.928464 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.928537 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.928509 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dg264\" (UniqueName: \"kubernetes.io/projected/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-kube-api-access-dg264\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.928609 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.928547 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-metrics-client-ca\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.928609 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.928592 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-federate-client-tls\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.928704 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.928624 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-secret-telemeter-client\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.929198 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.929169 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-serving-certs-ca-bundle\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.929343 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.929314 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.929465 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.929340 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-metrics-client-ca\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.931123 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.931096 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.931208 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.931180 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-secret-telemeter-client\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.931320 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.931300 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-telemeter-client-tls\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.931357 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.931316 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-federate-client-tls\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.937995 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.937975 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg264\" (UniqueName: \"kubernetes.io/projected/b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0-kube-api-access-dg264\") pod \"telemeter-client-5cb7d9d955-fw67s\" (UID: \"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0\") " pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:39.997572 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:39.997550 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" May 29 00:49:40.135241 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:40.135033 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cb7d9d955-fw67s"] May 29 00:49:40.216429 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:40.216393 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e3ebb6_faba_4aae_a9cc_1786cf72c2b0.slice/crio-776bd75b3185dbd00b5a9776c8940774d9a007aae6bcfdee7c8c079f03ee7c43 WatchSource:0}: Error finding container 776bd75b3185dbd00b5a9776c8940774d9a007aae6bcfdee7c8c079f03ee7c43: Status 404 returned error can't find the container with id 776bd75b3185dbd00b5a9776c8940774d9a007aae6bcfdee7c8c079f03ee7c43 May 29 00:49:40.570847 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:40.570806 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" event={"ID":"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0","Type":"ContainerStarted","Data":"776bd75b3185dbd00b5a9776c8940774d9a007aae6bcfdee7c8c079f03ee7c43"} May 29 00:49:40.573471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:40.573439 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" event={"ID":"96d65807-0a2f-4ecc-a509-454fc349c6de","Type":"ContainerStarted","Data":"5a10335d4a49b63f5409248e126a70dc86b675ca01596937724f822670db5fb3"} May 29 00:49:40.573471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:40.573472 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" event={"ID":"96d65807-0a2f-4ecc-a509-454fc349c6de","Type":"ContainerStarted","Data":"10d4b725a7d2de916ead71ab9eab48c7b309cd75354e19b3e794cf490bc3c219"} May 29 00:49:40.573621 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:40.573481 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" event={"ID":"96d65807-0a2f-4ecc-a509-454fc349c6de","Type":"ContainerStarted","Data":"8b337816a06e6ab26ce292ebbe7ad56c54f9e2947dd34805bca8eed4ffc0941a"} May 29 00:49:40.573659 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:40.573638 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:40.593866 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:40.593813 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" podStartSLOduration=1.321735497 podStartE2EDuration="4.593800259s" podCreationTimestamp="2026-05-29 00:49:36 +0000 UTC" firstStartedPulling="2026-05-29 00:49:36.988601875 +0000 UTC m=+135.448805307" lastFinishedPulling="2026-05-29 00:49:40.260666621 +0000 UTC m=+138.720870069" observedRunningTime="2026-05-29 00:49:40.592958496 +0000 UTC m=+139.053161946" watchObservedRunningTime="2026-05-29 00:49:40.593800259 +0000 UTC m=+139.054003709" May 29 00:49:42.584009 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:42.583980 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" event={"ID":"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0","Type":"ContainerStarted","Data":"b54a0b660b2b003aff89e8da6886cf814b2c8e6dcf7fd4dbbcfac494b3c8f445"} May 29 00:49:43.588974 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:43.588947 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" event={"ID":"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0","Type":"ContainerStarted","Data":"151a795a7eb3822f534b5a32851e75bf0427e4235c024c7214af32dc167db0ce"} May 29 00:49:43.589283 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:43.588979 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" event={"ID":"b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0","Type":"ContainerStarted","Data":"ab710f1577209e0d775624b85f10a72cd89d269ce252569090c7bec5a257986f"} May 29 00:49:43.610550 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:43.610508 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5cb7d9d955-fw67s" podStartSLOduration=1.46091408 podStartE2EDuration="4.610497044s" podCreationTimestamp="2026-05-29 00:49:39 +0000 UTC" firstStartedPulling="2026-05-29 00:49:40.218503926 +0000 UTC m=+138.678707355" lastFinishedPulling="2026-05-29 00:49:43.368086875 +0000 UTC m=+141.828290319" observedRunningTime="2026-05-29 00:49:43.609659342 +0000 UTC m=+142.069862804" watchObservedRunningTime="2026-05-29 00:49:43.610497044 +0000 UTC m=+142.070700494" May 29 00:49:46.587819 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:46.587788 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-55d6f9fc57-khts4" May 29 00:49:47.137254 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:47.137225 2551 scope.go:117] "RemoveContainer" containerID="29f6927340928a96a0f4cc2cf63c7d0b6d48050c4c200a29b412e32daaa091f9" May 29 00:49:47.602673 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:47.602649 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 00:49:47.603048 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:47.602728 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" event={"ID":"bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2","Type":"ContainerStarted","Data":"8e630a4d794e96ae9b77df07dcc9d2d08ea2e2aae574cb5af8008c073f5c5e65"} May 29 00:49:47.603048 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:47.603005 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:49:47.618267 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:47.618217 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" podStartSLOduration=55.350293474 podStartE2EDuration="57.61820422s" podCreationTimestamp="2026-05-29 00:48:50 +0000 UTC" firstStartedPulling="2026-05-29 00:48:51.162892228 +0000 UTC m=+89.623095656" lastFinishedPulling="2026-05-29 00:48:53.430802974 +0000 UTC m=+91.891006402" observedRunningTime="2026-05-29 00:49:47.617515821 +0000 UTC m=+146.077719274" watchObservedRunningTime="2026-05-29 00:49:47.61820422 +0000 UTC m=+146.078407670" May 29 00:49:48.011807 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.011777 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-m6z2w" May 29 00:49:48.200869 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.200835 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-fzcdb"] May 29 00:49:48.203313 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.203291 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-fzcdb" May 29 00:49:48.205976 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.205955 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 29 00:49:48.206064 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.206041 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 29 00:49:48.206139 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.206083 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hbcjh\"" May 29 00:49:48.213134 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.213116 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-fzcdb"] May 29 00:49:48.398452 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.398386 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kjsh\" (UniqueName: \"kubernetes.io/projected/9ecf3dfd-1802-4256-bc32-2b155f7ff18b-kube-api-access-8kjsh\") pod \"downloads-57bd96b888-fzcdb\" (UID: \"9ecf3dfd-1802-4256-bc32-2b155f7ff18b\") " pod="openshift-console/downloads-57bd96b888-fzcdb" May 29 00:49:48.499136 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.499107 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8kjsh\" (UniqueName: \"kubernetes.io/projected/9ecf3dfd-1802-4256-bc32-2b155f7ff18b-kube-api-access-8kjsh\") pod \"downloads-57bd96b888-fzcdb\" (UID: \"9ecf3dfd-1802-4256-bc32-2b155f7ff18b\") " pod="openshift-console/downloads-57bd96b888-fzcdb" May 29 00:49:48.526528 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.526502 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kjsh\" (UniqueName: \"kubernetes.io/projected/9ecf3dfd-1802-4256-bc32-2b155f7ff18b-kube-api-access-8kjsh\") pod \"downloads-57bd96b888-fzcdb\" (UID: \"9ecf3dfd-1802-4256-bc32-2b155f7ff18b\") " pod="openshift-console/downloads-57bd96b888-fzcdb" May 29 00:49:48.813410 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.813381 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-fzcdb" May 29 00:49:48.932726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:48.932705 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-fzcdb"] May 29 00:49:48.934588 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:48.934559 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ecf3dfd_1802_4256_bc32_2b155f7ff18b.slice/crio-5e6e21c5bc1765d9db0a0cb9ae7cf597694fb669749d22ec780220f8d6b4b5b8 WatchSource:0}: Error finding container 5e6e21c5bc1765d9db0a0cb9ae7cf597694fb669749d22ec780220f8d6b4b5b8: Status 404 returned error can't find the container with id 5e6e21c5bc1765d9db0a0cb9ae7cf597694fb669749d22ec780220f8d6b4b5b8 May 29 00:49:49.608613 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:49.608576 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-fzcdb" event={"ID":"9ecf3dfd-1802-4256-bc32-2b155f7ff18b","Type":"ContainerStarted","Data":"5e6e21c5bc1765d9db0a0cb9ae7cf597694fb669749d22ec780220f8d6b4b5b8"} May 29 00:49:57.313938 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.313906 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b589bbdf9-s24tp"] May 29 00:49:57.317195 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.317177 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.319840 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.319821 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 29 00:49:57.321061 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.321040 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 29 00:49:57.321159 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.321046 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wp4jf\"" May 29 00:49:57.321159 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.321097 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 29 00:49:57.321159 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.321105 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 29 00:49:57.321159 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.321098 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 29 00:49:57.326267 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.326243 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b589bbdf9-s24tp"] May 29 00:49:57.366069 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.366047 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-oauth-serving-cert\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.366188 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.366104 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-oauth-config\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.366188 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.366155 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-config\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.366275 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.366212 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-service-ca\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.366275 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.366266 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj9rz\" (UniqueName: \"kubernetes.io/projected/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-kube-api-access-nj9rz\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.366360 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.366305 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-serving-cert\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.466720 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.466695 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-oauth-config\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.466848 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.466726 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-config\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.466929 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.466866 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-service-ca\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.466994 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.466925 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nj9rz\" (UniqueName: \"kubernetes.io/projected/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-kube-api-access-nj9rz\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.466994 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.466962 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-serving-cert\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.467102 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.467008 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-oauth-serving-cert\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.467478 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.467449 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-config\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.467478 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.467472 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-service-ca\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.467631 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.467582 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-oauth-serving-cert\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.469512 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:57.469478 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-n5lpq" podUID="a0aa4161-0aa4-4e61-b55e-d737c3bfaecd" May 29 00:49:57.469609 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.469565 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-oauth-config\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.469899 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.469877 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-serving-cert\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.475610 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.475589 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj9rz\" (UniqueName: \"kubernetes.io/projected/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-kube-api-access-nj9rz\") pod \"console-6b589bbdf9-s24tp\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.483398 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:49:57.483368 2551 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-n5szr" podUID="bec90e87-7a28-4f14-83c6-61def84dbed9" May 29 00:49:57.627477 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.627416 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:49:57.634694 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.634670 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n5lpq" May 29 00:49:57.749712 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:57.749683 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b589bbdf9-s24tp"] May 29 00:49:57.753083 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:49:57.753053 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a27483_d9ca_4ba2_a8a7_89d1f0bda6c2.slice/crio-536b931d0642c48676b891d82e133dcc631f77480122c3d9897acaa43cb02625 WatchSource:0}: Error finding container 536b931d0642c48676b891d82e133dcc631f77480122c3d9897acaa43cb02625: Status 404 returned error can't find the container with id 536b931d0642c48676b891d82e133dcc631f77480122c3d9897acaa43cb02625 May 29 00:49:58.639251 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:49:58.639176 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b589bbdf9-s24tp" event={"ID":"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2","Type":"ContainerStarted","Data":"536b931d0642c48676b891d82e133dcc631f77480122c3d9897acaa43cb02625"} May 29 00:50:02.408280 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:02.408243 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:50:02.408735 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:02.408317 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:50:02.410798 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:02.410774 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0aa4161-0aa4-4e61-b55e-d737c3bfaecd-metrics-tls\") pod \"dns-default-n5lpq\" (UID: \"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd\") " pod="openshift-dns/dns-default-n5lpq" May 29 00:50:02.410970 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:02.410949 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bec90e87-7a28-4f14-83c6-61def84dbed9-cert\") pod \"ingress-canary-n5szr\" (UID: \"bec90e87-7a28-4f14-83c6-61def84dbed9\") " pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:50:02.438529 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:02.438508 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7vx5w\"" May 29 00:50:02.445833 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:02.445811 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n5lpq" May 29 00:50:05.697445 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.697369 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-64997455d5-z5q72"] May 29 00:50:05.702061 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.702036 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.711776 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.711196 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64997455d5-z5q72"] May 29 00:50:05.711989 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.711934 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 29 00:50:05.736692 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.736664 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42c5s\" (UniqueName: \"kubernetes.io/projected/22c215f4-d306-4895-8481-b566a53e6e51-kube-api-access-42c5s\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.736824 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.736703 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-serving-cert\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.736824 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.736788 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-trusted-ca-bundle\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.736921 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.736854 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-oauth-serving-cert\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.736921 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.736883 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-oauth-config\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.736921 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.736905 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-console-config\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.737065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.736950 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-service-ca\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.838008 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.837974 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-42c5s\" (UniqueName: \"kubernetes.io/projected/22c215f4-d306-4895-8481-b566a53e6e51-kube-api-access-42c5s\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.838201 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.838024 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-serving-cert\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.838201 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.838059 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-trusted-ca-bundle\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.838201 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.838103 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-oauth-serving-cert\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.838201 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.838130 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-oauth-config\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.838201 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.838154 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-console-config\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.838201 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.838191 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-service-ca\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.839180 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.839151 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-service-ca\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.839296 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.839200 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-trusted-ca-bundle\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.839427 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.839403 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-oauth-serving-cert\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.839513 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.839465 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-console-config\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.840923 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.840888 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-serving-cert\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.841026 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.840908 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-oauth-config\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:05.855190 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:05.855161 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42c5s\" (UniqueName: \"kubernetes.io/projected/22c215f4-d306-4895-8481-b566a53e6e51-kube-api-access-42c5s\") pod \"console-64997455d5-z5q72\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:06.014969 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:06.014935 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:07.004468 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.004444 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n5lpq"] May 29 00:50:07.007028 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:50:07.006994 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0aa4161_0aa4_4e61_b55e_d737c3bfaecd.slice/crio-ecdbbd13ec08f7bcb74b4a3a5c017011ef693dc57fe00a61963f13660e4b1b0b WatchSource:0}: Error finding container ecdbbd13ec08f7bcb74b4a3a5c017011ef693dc57fe00a61963f13660e4b1b0b: Status 404 returned error can't find the container with id ecdbbd13ec08f7bcb74b4a3a5c017011ef693dc57fe00a61963f13660e4b1b0b May 29 00:50:07.022980 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.022957 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64997455d5-z5q72"] May 29 00:50:07.026083 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:50:07.026058 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c215f4_d306_4895_8481_b566a53e6e51.slice/crio-9c5a2a3d04c351e8cd88426aef06892c56540d553dee02d52b745bef58306cae WatchSource:0}: Error finding container 9c5a2a3d04c351e8cd88426aef06892c56540d553dee02d52b745bef58306cae: Status 404 returned error can't find the container with id 9c5a2a3d04c351e8cd88426aef06892c56540d553dee02d52b745bef58306cae May 29 00:50:07.666902 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.666863 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-fzcdb" event={"ID":"9ecf3dfd-1802-4256-bc32-2b155f7ff18b","Type":"ContainerStarted","Data":"fd5c17d3d7118544827cafb63fa273bfc561ed138bf59e529c5f714458d5b2ec"} May 29 00:50:07.667264 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.667243 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-fzcdb" May 29 00:50:07.668792 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.668740 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b589bbdf9-s24tp" event={"ID":"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2","Type":"ContainerStarted","Data":"57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5"} May 29 00:50:07.670596 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.670568 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64997455d5-z5q72" event={"ID":"22c215f4-d306-4895-8481-b566a53e6e51","Type":"ContainerStarted","Data":"83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d"} May 29 00:50:07.670720 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.670603 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64997455d5-z5q72" event={"ID":"22c215f4-d306-4895-8481-b566a53e6e51","Type":"ContainerStarted","Data":"9c5a2a3d04c351e8cd88426aef06892c56540d553dee02d52b745bef58306cae"} May 29 00:50:07.671596 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.671572 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n5lpq" event={"ID":"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd","Type":"ContainerStarted","Data":"ecdbbd13ec08f7bcb74b4a3a5c017011ef693dc57fe00a61963f13660e4b1b0b"} May 29 00:50:07.679201 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.679179 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-fzcdb" May 29 00:50:07.686620 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.686284 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-fzcdb" podStartSLOduration=1.697003553 podStartE2EDuration="19.686268673s" podCreationTimestamp="2026-05-29 00:49:48 +0000 UTC" firstStartedPulling="2026-05-29 00:49:48.936372301 +0000 UTC m=+147.396575728" lastFinishedPulling="2026-05-29 00:50:06.925637405 +0000 UTC m=+165.385840848" observedRunningTime="2026-05-29 00:50:07.683247397 +0000 UTC m=+166.143450859" watchObservedRunningTime="2026-05-29 00:50:07.686268673 +0000 UTC m=+166.146472125" May 29 00:50:07.700050 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.700003 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6b589bbdf9-s24tp" podStartSLOduration=1.577750368 podStartE2EDuration="10.699989416s" podCreationTimestamp="2026-05-29 00:49:57 +0000 UTC" firstStartedPulling="2026-05-29 00:49:57.755256089 +0000 UTC m=+156.215459521" lastFinishedPulling="2026-05-29 00:50:06.877495128 +0000 UTC m=+165.337698569" observedRunningTime="2026-05-29 00:50:07.698698038 +0000 UTC m=+166.158901490" watchObservedRunningTime="2026-05-29 00:50:07.699989416 +0000 UTC m=+166.160192867" May 29 00:50:07.719633 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:07.719588 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-64997455d5-z5q72" podStartSLOduration=2.7195783479999998 podStartE2EDuration="2.719578348s" podCreationTimestamp="2026-05-29 00:50:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 00:50:07.718440046 +0000 UTC m=+166.178643497" watchObservedRunningTime="2026-05-29 00:50:07.719578348 +0000 UTC m=+166.179781800" May 29 00:50:10.690835 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:10.690796 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n5lpq" event={"ID":"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd","Type":"ContainerStarted","Data":"a24ab73d27cefd5752bce5863ad42cadcc717e7c4165e45b56f9c58225f05b2a"} May 29 00:50:10.690835 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:10.690837 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n5lpq" event={"ID":"a0aa4161-0aa4-4e61-b55e-d737c3bfaecd","Type":"ContainerStarted","Data":"d56bb71076056f9bf4117f00bd2a6c8969ffb87ca390d1a9be2e98574bc348d1"} May 29 00:50:10.691383 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:10.690880 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n5lpq" May 29 00:50:10.708930 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:10.708882 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-n5lpq" podStartSLOduration=134.183721534 podStartE2EDuration="2m16.70886595s" podCreationTimestamp="2026-05-29 00:47:54 +0000 UTC" firstStartedPulling="2026-05-29 00:50:07.00982407 +0000 UTC m=+165.470027499" lastFinishedPulling="2026-05-29 00:50:09.534968474 +0000 UTC m=+167.995171915" observedRunningTime="2026-05-29 00:50:10.705911074 +0000 UTC m=+169.166114526" watchObservedRunningTime="2026-05-29 00:50:10.70886595 +0000 UTC m=+169.169069400" May 29 00:50:11.137258 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:11.137224 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:50:11.140850 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:11.140824 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nghjs\"" May 29 00:50:11.148070 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:11.148045 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5szr" May 29 00:50:11.292345 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:11.292316 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n5szr"] May 29 00:50:11.294985 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:50:11.294950 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec90e87_7a28_4f14_83c6_61def84dbed9.slice/crio-7aba3a64f5e941a051e97f3bf8e1af6b35fdd45730186daf87ab5a7355108f9b WatchSource:0}: Error finding container 7aba3a64f5e941a051e97f3bf8e1af6b35fdd45730186daf87ab5a7355108f9b: Status 404 returned error can't find the container with id 7aba3a64f5e941a051e97f3bf8e1af6b35fdd45730186daf87ab5a7355108f9b May 29 00:50:11.695945 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:11.695901 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n5szr" event={"ID":"bec90e87-7a28-4f14-83c6-61def84dbed9","Type":"ContainerStarted","Data":"7aba3a64f5e941a051e97f3bf8e1af6b35fdd45730186daf87ab5a7355108f9b"} May 29 00:50:14.707134 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:14.707091 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n5szr" event={"ID":"bec90e87-7a28-4f14-83c6-61def84dbed9","Type":"ContainerStarted","Data":"6d25082aa548bab5b4b9ad815689432e619a0a0bccebffa0fadf7734f37556bf"} May 29 00:50:14.725218 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:14.725158 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n5szr" podStartSLOduration=138.30472632 podStartE2EDuration="2m20.725130836s" podCreationTimestamp="2026-05-29 00:47:54 +0000 UTC" firstStartedPulling="2026-05-29 00:50:11.297327348 +0000 UTC m=+169.757530793" lastFinishedPulling="2026-05-29 00:50:13.717731868 +0000 UTC m=+172.177935309" observedRunningTime="2026-05-29 00:50:14.72458277 +0000 UTC m=+173.184786220" watchObservedRunningTime="2026-05-29 00:50:14.725130836 +0000 UTC m=+173.185334282" May 29 00:50:16.015690 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:16.015651 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:16.015690 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:16.015694 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:16.016981 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:16.016958 2551 patch_prober.go:28] interesting pod/console-64997455d5-z5q72 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.19:8443/health\": dial tcp 10.134.0.19:8443: connect: connection refused" start-of-body= May 29 00:50:16.017043 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:16.017003 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-64997455d5-z5q72" podUID="22c215f4-d306-4895-8481-b566a53e6e51" containerName="console" probeResult="failure" output="Get \"https://10.134.0.19:8443/health\": dial tcp 10.134.0.19:8443: connect: connection refused" May 29 00:50:17.628037 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:17.627998 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:50:17.628864 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:17.628049 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:50:17.632575 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:17.632553 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:50:17.724173 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:17.724147 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:50:20.698979 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:20.698942 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n5lpq" May 29 00:50:21.730386 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:21.730354 2551 generic.go:358] "Generic (PLEG): container finished" podID="14667b39-73c4-45e6-a9e1-d7fb935361e2" containerID="f7352e9a992b7ecbf887d3ef4bb34cb105852afda025ed18543fd3f0d11abae0" exitCode=255 May 29 00:50:21.730791 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:21.730414 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" event={"ID":"14667b39-73c4-45e6-a9e1-d7fb935361e2","Type":"ContainerDied","Data":"f7352e9a992b7ecbf887d3ef4bb34cb105852afda025ed18543fd3f0d11abae0"} May 29 00:50:21.736515 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:21.736495 2551 scope.go:117] "RemoveContainer" containerID="f7352e9a992b7ecbf887d3ef4bb34cb105852afda025ed18543fd3f0d11abae0" May 29 00:50:22.734576 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:22.734546 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8676556566-wsm6n" event={"ID":"14667b39-73c4-45e6-a9e1-d7fb935361e2","Type":"ContainerStarted","Data":"3f7e0507a447adcb4a0acf6a5f95546853940116d3e50d7844389f290b6ff861"} May 29 00:50:26.019854 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:26.019825 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:26.023779 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:26.023742 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64997455d5-z5q72" May 29 00:50:26.078704 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:26.078679 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b589bbdf9-s24tp"] May 29 00:50:51.098904 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.098849 2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6b589bbdf9-s24tp" podUID="e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" containerName="console" containerID="cri-o://57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5" gracePeriod=15 May 29 00:50:51.367946 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.367925 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b589bbdf9-s24tp_e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2/console/0.log" May 29 00:50:51.368046 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.367986 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:50:51.497420 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497388 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-serving-cert\") pod \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " May 29 00:50:51.497597 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497427 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-config\") pod \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " May 29 00:50:51.497597 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497447 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-oauth-serving-cert\") pod \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " May 29 00:50:51.497597 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497493 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-oauth-config\") pod \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " May 29 00:50:51.497597 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497532 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj9rz\" (UniqueName: \"kubernetes.io/projected/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-kube-api-access-nj9rz\") pod \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " May 29 00:50:51.497597 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497554 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-service-ca\") pod \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\" (UID: \"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2\") " May 29 00:50:51.497949 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497918 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-config" (OuterVolumeSpecName: "console-config") pod "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" (UID: "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:50:51.498011 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497959 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" (UID: "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:50:51.498011 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.497979 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-service-ca" (OuterVolumeSpecName: "service-ca") pod "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" (UID: "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:50:51.499785 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.499744 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" (UID: "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 00:50:51.499896 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.499814 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-kube-api-access-nj9rz" (OuterVolumeSpecName: "kube-api-access-nj9rz") pod "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" (UID: "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2"). InnerVolumeSpecName "kube-api-access-nj9rz". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 00:50:51.500059 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.500036 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" (UID: "e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 00:50:51.598086 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.598066 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-service-ca\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:50:51.598086 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.598087 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-serving-cert\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:50:51.598228 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.598097 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-config\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:50:51.598228 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.598106 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-oauth-serving-cert\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:50:51.598228 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.598116 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-console-oauth-config\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:50:51.598228 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.598125 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj9rz\" (UniqueName: \"kubernetes.io/projected/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2-kube-api-access-nj9rz\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:50:51.816242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.816223 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b589bbdf9-s24tp_e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2/console/0.log" May 29 00:50:51.816340 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.816258 2551 generic.go:358] "Generic (PLEG): container finished" podID="e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" containerID="57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5" exitCode=2 May 29 00:50:51.816340 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.816308 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b589bbdf9-s24tp" event={"ID":"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2","Type":"ContainerDied","Data":"57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5"} May 29 00:50:51.816340 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.816330 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b589bbdf9-s24tp" event={"ID":"e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2","Type":"ContainerDied","Data":"536b931d0642c48676b891d82e133dcc631f77480122c3d9897acaa43cb02625"} May 29 00:50:51.816340 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.816328 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b589bbdf9-s24tp" May 29 00:50:51.816467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.816400 2551 scope.go:117] "RemoveContainer" containerID="57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5" May 29 00:50:51.829530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.829513 2551 scope.go:117] "RemoveContainer" containerID="57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5" May 29 00:50:51.829801 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:50:51.829777 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5\": container with ID starting with 57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5 not found: ID does not exist" containerID="57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5" May 29 00:50:51.829886 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.829809 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5"} err="failed to get container status \"57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5\": rpc error: code = NotFound desc = could not find container \"57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5\": container with ID starting with 57ff0b1980e51e507ed25f2705afd5df3d630b025fd5a3a1ffee0367936a4bb5 not found: ID does not exist" May 29 00:50:51.836404 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.836383 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b589bbdf9-s24tp"] May 29 00:50:51.840212 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:51.840195 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b589bbdf9-s24tp"] May 29 00:50:52.140471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:50:52.140408 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" path="/var/lib/kubelet/pods/e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2/volumes" May 29 00:51:13.821720 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.821677 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-66f9f94d8f-kbm94"] May 29 00:51:13.822214 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.822075 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" containerName="console" May 29 00:51:13.822214 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.822091 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" containerName="console" May 29 00:51:13.822214 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.822194 2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="e8a27483-d9ca-4ba2-a8a7-89d1f0bda6c2" containerName="console" May 29 00:51:13.824408 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.824387 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:13.835272 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.835252 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66f9f94d8f-kbm94"] May 29 00:51:13.952359 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.952329 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-trusted-ca-bundle\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:13.952359 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.952361 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-oauth-config\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:13.952525 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.952377 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-config\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:13.952525 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.952474 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-service-ca\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:13.952592 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.952532 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpvm4\" (UniqueName: \"kubernetes.io/projected/a02d348b-9e24-4ea7-b372-1e9d19f7507b-kube-api-access-jpvm4\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:13.952592 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.952562 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-serving-cert\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:13.952592 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:13.952585 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-oauth-serving-cert\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.053444 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.053417 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-service-ca\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.053550 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.053455 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jpvm4\" (UniqueName: \"kubernetes.io/projected/a02d348b-9e24-4ea7-b372-1e9d19f7507b-kube-api-access-jpvm4\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.053550 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.053477 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-serving-cert\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.053629 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.053587 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-oauth-serving-cert\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.053674 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.053642 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-trusted-ca-bundle\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.053724 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.053671 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-oauth-config\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.053724 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.053696 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-config\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.054192 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.054159 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-service-ca\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.054363 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.054342 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-oauth-serving-cert\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.054428 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.054367 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-config\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.054534 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.054513 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-trusted-ca-bundle\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.056042 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.056023 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-serving-cert\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.056154 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.056136 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-oauth-config\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.060794 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.060774 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpvm4\" (UniqueName: \"kubernetes.io/projected/a02d348b-9e24-4ea7-b372-1e9d19f7507b-kube-api-access-jpvm4\") pod \"console-66f9f94d8f-kbm94\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.134743 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.134680 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:14.257980 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.257949 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66f9f94d8f-kbm94"] May 29 00:51:14.262020 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:51:14.261993 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02d348b_9e24_4ea7_b372_1e9d19f7507b.slice/crio-d6154c99880f90644116c20b64f695e11a16272cd3c96f87abd0c7464e6e49d0 WatchSource:0}: Error finding container d6154c99880f90644116c20b64f695e11a16272cd3c96f87abd0c7464e6e49d0: Status 404 returned error can't find the container with id d6154c99880f90644116c20b64f695e11a16272cd3c96f87abd0c7464e6e49d0 May 29 00:51:14.884168 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.884126 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f9f94d8f-kbm94" event={"ID":"a02d348b-9e24-4ea7-b372-1e9d19f7507b","Type":"ContainerStarted","Data":"ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99"} May 29 00:51:14.884526 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.884174 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f9f94d8f-kbm94" event={"ID":"a02d348b-9e24-4ea7-b372-1e9d19f7507b","Type":"ContainerStarted","Data":"d6154c99880f90644116c20b64f695e11a16272cd3c96f87abd0c7464e6e49d0"} May 29 00:51:14.901957 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:14.901914 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-66f9f94d8f-kbm94" podStartSLOduration=1.901900394 podStartE2EDuration="1.901900394s" podCreationTimestamp="2026-05-29 00:51:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 00:51:14.900581448 +0000 UTC m=+233.360784900" watchObservedRunningTime="2026-05-29 00:51:14.901900394 +0000 UTC m=+233.362103843" May 29 00:51:24.135567 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:24.135531 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:24.136055 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:24.135575 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:24.141365 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:24.141340 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:24.914721 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:24.914688 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:51:24.954819 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:24.954772 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64997455d5-z5q72"] May 29 00:51:25.442458 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.442428 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vgjjf"] May 29 00:51:25.445570 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.445553 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.448484 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.448464 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 29 00:51:25.452606 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.452584 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vgjjf"] May 29 00:51:25.537661 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.537633 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47a98844-8ffd-4487-b470-f3d17d86446d-dbus\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.537787 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.537673 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47a98844-8ffd-4487-b470-f3d17d86446d-original-pull-secret\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.537845 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.537781 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47a98844-8ffd-4487-b470-f3d17d86446d-kubelet-config\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.639050 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.639024 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47a98844-8ffd-4487-b470-f3d17d86446d-kubelet-config\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.639139 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.639072 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47a98844-8ffd-4487-b470-f3d17d86446d-dbus\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.639192 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.639136 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47a98844-8ffd-4487-b470-f3d17d86446d-kubelet-config\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.639192 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.639183 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47a98844-8ffd-4487-b470-f3d17d86446d-original-pull-secret\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.639279 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.639217 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47a98844-8ffd-4487-b470-f3d17d86446d-dbus\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.641270 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.641252 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47a98844-8ffd-4487-b470-f3d17d86446d-original-pull-secret\") pod \"global-pull-secret-syncer-vgjjf\" (UID: \"47a98844-8ffd-4487-b470-f3d17d86446d\") " pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.755065 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.755035 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vgjjf" May 29 00:51:25.873020 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.872980 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vgjjf"] May 29 00:51:25.875956 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:51:25.875930 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a98844_8ffd_4487_b470_f3d17d86446d.slice/crio-e6fb127070c0fb4ba677ddb23a9443efa4c8ec2fcc93b8f2b10ff3fe89ceb8b1 WatchSource:0}: Error finding container e6fb127070c0fb4ba677ddb23a9443efa4c8ec2fcc93b8f2b10ff3fe89ceb8b1: Status 404 returned error can't find the container with id e6fb127070c0fb4ba677ddb23a9443efa4c8ec2fcc93b8f2b10ff3fe89ceb8b1 May 29 00:51:25.918403 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:25.918377 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vgjjf" event={"ID":"47a98844-8ffd-4487-b470-f3d17d86446d","Type":"ContainerStarted","Data":"e6fb127070c0fb4ba677ddb23a9443efa4c8ec2fcc93b8f2b10ff3fe89ceb8b1"} May 29 00:51:32.940568 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:32.940527 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vgjjf" event={"ID":"47a98844-8ffd-4487-b470-f3d17d86446d","Type":"ContainerStarted","Data":"208717b0211319023b5fe3c82acfe32fd3839dfecb955f2abe7328fd474d41d3"} May 29 00:51:32.955910 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:32.955865 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vgjjf" podStartSLOduration=1.789645148 podStartE2EDuration="7.95585196s" podCreationTimestamp="2026-05-29 00:51:25 +0000 UTC" firstStartedPulling="2026-05-29 00:51:25.877511221 +0000 UTC m=+244.337714650" lastFinishedPulling="2026-05-29 00:51:32.043718031 +0000 UTC m=+250.503921462" observedRunningTime="2026-05-29 00:51:32.954644892 +0000 UTC m=+251.414848341" watchObservedRunningTime="2026-05-29 00:51:32.95585196 +0000 UTC m=+251.416055428" May 29 00:51:49.977446 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:49.977337 2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-64997455d5-z5q72" podUID="22c215f4-d306-4895-8481-b566a53e6e51" containerName="console" containerID="cri-o://83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d" gracePeriod=15 May 29 00:51:50.215729 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.215709 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64997455d5-z5q72_22c215f4-d306-4895-8481-b566a53e6e51/console/0.log" May 29 00:51:50.215839 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.215784 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64997455d5-z5q72" May 29 00:51:50.325531 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325455 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-console-config\") pod \"22c215f4-d306-4895-8481-b566a53e6e51\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " May 29 00:51:50.325531 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325498 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-oauth-serving-cert\") pod \"22c215f4-d306-4895-8481-b566a53e6e51\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " May 29 00:51:50.325531 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325537 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-trusted-ca-bundle\") pod \"22c215f4-d306-4895-8481-b566a53e6e51\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " May 29 00:51:50.325797 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325639 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-oauth-config\") pod \"22c215f4-d306-4895-8481-b566a53e6e51\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " May 29 00:51:50.325797 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325689 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42c5s\" (UniqueName: \"kubernetes.io/projected/22c215f4-d306-4895-8481-b566a53e6e51-kube-api-access-42c5s\") pod \"22c215f4-d306-4895-8481-b566a53e6e51\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " May 29 00:51:50.325901 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325821 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-serving-cert\") pod \"22c215f4-d306-4895-8481-b566a53e6e51\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " May 29 00:51:50.325901 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325872 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-service-ca\") pod \"22c215f4-d306-4895-8481-b566a53e6e51\" (UID: \"22c215f4-d306-4895-8481-b566a53e6e51\") " May 29 00:51:50.326022 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325893 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-console-config" (OuterVolumeSpecName: "console-config") pod "22c215f4-d306-4895-8481-b566a53e6e51" (UID: "22c215f4-d306-4895-8481-b566a53e6e51"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:51:50.326022 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325906 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "22c215f4-d306-4895-8481-b566a53e6e51" (UID: "22c215f4-d306-4895-8481-b566a53e6e51"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:51:50.326022 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.325956 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "22c215f4-d306-4895-8481-b566a53e6e51" (UID: "22c215f4-d306-4895-8481-b566a53e6e51"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:51:50.326169 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.326092 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-console-config\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:51:50.326169 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.326112 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-oauth-serving-cert\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:51:50.326169 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.326127 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-trusted-ca-bundle\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:51:50.326313 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.326275 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-service-ca" (OuterVolumeSpecName: "service-ca") pod "22c215f4-d306-4895-8481-b566a53e6e51" (UID: "22c215f4-d306-4895-8481-b566a53e6e51"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:51:50.327862 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.327841 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "22c215f4-d306-4895-8481-b566a53e6e51" (UID: "22c215f4-d306-4895-8481-b566a53e6e51"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 00:51:50.328148 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.328124 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "22c215f4-d306-4895-8481-b566a53e6e51" (UID: "22c215f4-d306-4895-8481-b566a53e6e51"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 00:51:50.328237 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.328181 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c215f4-d306-4895-8481-b566a53e6e51-kube-api-access-42c5s" (OuterVolumeSpecName: "kube-api-access-42c5s") pod "22c215f4-d306-4895-8481-b566a53e6e51" (UID: "22c215f4-d306-4895-8481-b566a53e6e51"). InnerVolumeSpecName "kube-api-access-42c5s". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 00:51:50.427076 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.427056 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42c5s\" (UniqueName: \"kubernetes.io/projected/22c215f4-d306-4895-8481-b566a53e6e51-kube-api-access-42c5s\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:51:50.427076 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.427076 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-serving-cert\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:51:50.427188 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.427087 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22c215f4-d306-4895-8481-b566a53e6e51-service-ca\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:51:50.427188 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.427097 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22c215f4-d306-4895-8481-b566a53e6e51-console-oauth-config\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:51:50.993165 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.993139 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64997455d5-z5q72_22c215f4-d306-4895-8481-b566a53e6e51/console/0.log" May 29 00:51:50.993610 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.993177 2551 generic.go:358] "Generic (PLEG): container finished" podID="22c215f4-d306-4895-8481-b566a53e6e51" containerID="83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d" exitCode=2 May 29 00:51:50.993610 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.993206 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64997455d5-z5q72" event={"ID":"22c215f4-d306-4895-8481-b566a53e6e51","Type":"ContainerDied","Data":"83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d"} May 29 00:51:50.993610 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.993226 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64997455d5-z5q72" event={"ID":"22c215f4-d306-4895-8481-b566a53e6e51","Type":"ContainerDied","Data":"9c5a2a3d04c351e8cd88426aef06892c56540d553dee02d52b745bef58306cae"} May 29 00:51:50.993610 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.993240 2551 scope.go:117] "RemoveContainer" containerID="83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d" May 29 00:51:50.993610 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:50.993259 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64997455d5-z5q72" May 29 00:51:51.001560 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:51.001544 2551 scope.go:117] "RemoveContainer" containerID="83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d" May 29 00:51:51.001804 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:51:51.001780 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d\": container with ID starting with 83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d not found: ID does not exist" containerID="83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d" May 29 00:51:51.001894 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:51.001809 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d"} err="failed to get container status \"83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d\": rpc error: code = NotFound desc = could not find container \"83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d\": container with ID starting with 83667c966fbcb4034e18d1d19e2f77d502599fedf254986273dab90ed979f05d not found: ID does not exist" May 29 00:51:51.014155 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:51.014131 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64997455d5-z5q72"] May 29 00:51:51.019175 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:51.019157 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64997455d5-z5q72"] May 29 00:51:52.141121 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:51:52.141084 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22c215f4-d306-4895-8481-b566a53e6e51" path="/var/lib/kubelet/pods/22c215f4-d306-4895-8481-b566a53e6e51/volumes" May 29 00:52:05.591500 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.591463 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj"] May 29 00:52:05.592168 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.591813 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22c215f4-d306-4895-8481-b566a53e6e51" containerName="console" May 29 00:52:05.592168 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.591825 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="22c215f4-d306-4895-8481-b566a53e6e51" containerName="console" May 29 00:52:05.592168 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.591880 2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="22c215f4-d306-4895-8481-b566a53e6e51" containerName="console" May 29 00:52:05.593954 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.593935 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" May 29 00:52:05.596673 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.596632 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" May 29 00:52:05.596853 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.596676 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" May 29 00:52:05.596853 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.596650 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-74td7\"" May 29 00:52:05.605308 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.605281 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj"] May 29 00:52:05.736727 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.736704 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmvjs\" (UniqueName: \"kubernetes.io/projected/024e6979-ee26-441f-9fb9-df753679b61a-kube-api-access-tmvjs\") pod \"cert-manager-operator-controller-manager-54b9655956-d2lnj\" (UID: \"024e6979-ee26-441f-9fb9-df753679b61a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" May 29 00:52:05.736849 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.736737 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/024e6979-ee26-441f-9fb9-df753679b61a-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-d2lnj\" (UID: \"024e6979-ee26-441f-9fb9-df753679b61a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" May 29 00:52:05.837933 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.837905 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tmvjs\" (UniqueName: \"kubernetes.io/projected/024e6979-ee26-441f-9fb9-df753679b61a-kube-api-access-tmvjs\") pod \"cert-manager-operator-controller-manager-54b9655956-d2lnj\" (UID: \"024e6979-ee26-441f-9fb9-df753679b61a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" May 29 00:52:05.838036 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.837943 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/024e6979-ee26-441f-9fb9-df753679b61a-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-d2lnj\" (UID: \"024e6979-ee26-441f-9fb9-df753679b61a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" May 29 00:52:05.838255 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.838240 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/024e6979-ee26-441f-9fb9-df753679b61a-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-d2lnj\" (UID: \"024e6979-ee26-441f-9fb9-df753679b61a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" May 29 00:52:05.845779 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.845719 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmvjs\" (UniqueName: \"kubernetes.io/projected/024e6979-ee26-441f-9fb9-df753679b61a-kube-api-access-tmvjs\") pod \"cert-manager-operator-controller-manager-54b9655956-d2lnj\" (UID: \"024e6979-ee26-441f-9fb9-df753679b61a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" May 29 00:52:05.904683 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:05.904666 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" May 29 00:52:06.039584 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:06.039552 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj"] May 29 00:52:06.042379 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:52:06.042353 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024e6979_ee26_441f_9fb9_df753679b61a.slice/crio-2d1cacbeec5a80e666da5f38a4d3c70804b6569883fc35b2a04b40c0c184206b WatchSource:0}: Error finding container 2d1cacbeec5a80e666da5f38a4d3c70804b6569883fc35b2a04b40c0c184206b: Status 404 returned error can't find the container with id 2d1cacbeec5a80e666da5f38a4d3c70804b6569883fc35b2a04b40c0c184206b May 29 00:52:07.042337 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:07.042300 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" event={"ID":"024e6979-ee26-441f-9fb9-df753679b61a","Type":"ContainerStarted","Data":"2d1cacbeec5a80e666da5f38a4d3c70804b6569883fc35b2a04b40c0c184206b"} May 29 00:52:09.051436 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:09.051403 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" event={"ID":"024e6979-ee26-441f-9fb9-df753679b61a","Type":"ContainerStarted","Data":"419e6a946cc0869f4ba02e875bee09a2fa6c05732e91c584e764a0032bd6e46f"} May 29 00:52:09.071675 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:09.071633 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-d2lnj" podStartSLOduration=1.453021463 podStartE2EDuration="4.071593362s" podCreationTimestamp="2026-05-29 00:52:05 +0000 UTC" firstStartedPulling="2026-05-29 00:52:06.044726327 +0000 UTC m=+284.504929754" lastFinishedPulling="2026-05-29 00:52:08.663298222 +0000 UTC m=+287.123501653" observedRunningTime="2026-05-29 00:52:09.070202075 +0000 UTC m=+287.530405505" watchObservedRunningTime="2026-05-29 00:52:09.071593362 +0000 UTC m=+287.531796813" May 29 00:52:12.099418 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.099363 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-zs5sj"] May 29 00:52:12.101966 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.101948 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:12.104636 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.104615 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 29 00:52:12.104771 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.104698 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 29 00:52:12.105844 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.105824 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-6tsm5\"" May 29 00:52:12.112328 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.112305 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-zs5sj"] May 29 00:52:12.181502 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.181475 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6af778e-dabb-4838-936f-05b6a81a608d-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-zs5sj\" (UID: \"e6af778e-dabb-4838-936f-05b6a81a608d\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:12.181615 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.181521 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v698s\" (UniqueName: \"kubernetes.io/projected/e6af778e-dabb-4838-936f-05b6a81a608d-kube-api-access-v698s\") pod \"cert-manager-webhook-587ccfb98-zs5sj\" (UID: \"e6af778e-dabb-4838-936f-05b6a81a608d\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:12.282317 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.282295 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6af778e-dabb-4838-936f-05b6a81a608d-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-zs5sj\" (UID: \"e6af778e-dabb-4838-936f-05b6a81a608d\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:12.282412 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.282328 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v698s\" (UniqueName: \"kubernetes.io/projected/e6af778e-dabb-4838-936f-05b6a81a608d-kube-api-access-v698s\") pod \"cert-manager-webhook-587ccfb98-zs5sj\" (UID: \"e6af778e-dabb-4838-936f-05b6a81a608d\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:12.289926 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.289902 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6af778e-dabb-4838-936f-05b6a81a608d-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-zs5sj\" (UID: \"e6af778e-dabb-4838-936f-05b6a81a608d\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:12.290072 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.290056 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v698s\" (UniqueName: \"kubernetes.io/projected/e6af778e-dabb-4838-936f-05b6a81a608d-kube-api-access-v698s\") pod \"cert-manager-webhook-587ccfb98-zs5sj\" (UID: \"e6af778e-dabb-4838-936f-05b6a81a608d\") " pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:12.411167 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.411110 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:12.533023 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:12.532999 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-zs5sj"] May 29 00:52:12.534982 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:52:12.534949 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6af778e_dabb_4838_936f_05b6a81a608d.slice/crio-114c2394cd2176cb910d16c034bcd7f9a9241df308c064d7d308a688ab62d9f1 WatchSource:0}: Error finding container 114c2394cd2176cb910d16c034bcd7f9a9241df308c064d7d308a688ab62d9f1: Status 404 returned error can't find the container with id 114c2394cd2176cb910d16c034bcd7f9a9241df308c064d7d308a688ab62d9f1 May 29 00:52:13.053276 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.053246 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-t6kgt"] May 29 00:52:13.056299 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.056282 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" May 29 00:52:13.058973 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.058953 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-q887j\"" May 29 00:52:13.065453 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.065432 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-t6kgt"] May 29 00:52:13.067618 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.067595 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" event={"ID":"e6af778e-dabb-4838-936f-05b6a81a608d","Type":"ContainerStarted","Data":"114c2394cd2176cb910d16c034bcd7f9a9241df308c064d7d308a688ab62d9f1"} May 29 00:52:13.189533 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.189506 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9f55a2e-3b8d-463c-bfd7-ae11a935933c-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-t6kgt\" (UID: \"f9f55a2e-3b8d-463c-bfd7-ae11a935933c\") " pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" May 29 00:52:13.189884 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.189551 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trp8v\" (UniqueName: \"kubernetes.io/projected/f9f55a2e-3b8d-463c-bfd7-ae11a935933c-kube-api-access-trp8v\") pod \"cert-manager-cainjector-68b757865b-t6kgt\" (UID: \"f9f55a2e-3b8d-463c-bfd7-ae11a935933c\") " pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" May 29 00:52:13.290574 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.290548 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9f55a2e-3b8d-463c-bfd7-ae11a935933c-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-t6kgt\" (UID: \"f9f55a2e-3b8d-463c-bfd7-ae11a935933c\") " pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" May 29 00:52:13.290725 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.290610 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-trp8v\" (UniqueName: \"kubernetes.io/projected/f9f55a2e-3b8d-463c-bfd7-ae11a935933c-kube-api-access-trp8v\") pod \"cert-manager-cainjector-68b757865b-t6kgt\" (UID: \"f9f55a2e-3b8d-463c-bfd7-ae11a935933c\") " pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" May 29 00:52:13.298127 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.298105 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9f55a2e-3b8d-463c-bfd7-ae11a935933c-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-t6kgt\" (UID: \"f9f55a2e-3b8d-463c-bfd7-ae11a935933c\") " pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" May 29 00:52:13.298230 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.298145 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trp8v\" (UniqueName: \"kubernetes.io/projected/f9f55a2e-3b8d-463c-bfd7-ae11a935933c-kube-api-access-trp8v\") pod \"cert-manager-cainjector-68b757865b-t6kgt\" (UID: \"f9f55a2e-3b8d-463c-bfd7-ae11a935933c\") " pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" May 29 00:52:13.366799 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.366708 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" May 29 00:52:13.507221 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:13.507185 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-t6kgt"] May 29 00:52:13.511186 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:52:13.511156 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9f55a2e_3b8d_463c_bfd7_ae11a935933c.slice/crio-3dce03d21ba15600f7e4fc12cc3f05971c7eb2c5dcd1ae1535c1f915d5da93b1 WatchSource:0}: Error finding container 3dce03d21ba15600f7e4fc12cc3f05971c7eb2c5dcd1ae1535c1f915d5da93b1: Status 404 returned error can't find the container with id 3dce03d21ba15600f7e4fc12cc3f05971c7eb2c5dcd1ae1535c1f915d5da93b1 May 29 00:52:14.074300 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:14.074264 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" event={"ID":"f9f55a2e-3b8d-463c-bfd7-ae11a935933c","Type":"ContainerStarted","Data":"3dce03d21ba15600f7e4fc12cc3f05971c7eb2c5dcd1ae1535c1f915d5da93b1"} May 29 00:52:16.084198 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:16.084152 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" event={"ID":"e6af778e-dabb-4838-936f-05b6a81a608d","Type":"ContainerStarted","Data":"12053ae6a9d42cfd8039e4394f1a2ed4bb68275f6dae335998bea1b1a2ac8d74"} May 29 00:52:16.084626 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:16.084210 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:16.085474 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:16.085450 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" event={"ID":"f9f55a2e-3b8d-463c-bfd7-ae11a935933c","Type":"ContainerStarted","Data":"f182b9812472f82ab11875eda0ca33bd57aeb59093ef05cf96d4a334c55da249"} May 29 00:52:16.100208 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:16.100164 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" podStartSLOduration=1.522004613 podStartE2EDuration="4.100152365s" podCreationTimestamp="2026-05-29 00:52:12 +0000 UTC" firstStartedPulling="2026-05-29 00:52:12.536887124 +0000 UTC m=+290.997090552" lastFinishedPulling="2026-05-29 00:52:15.115034876 +0000 UTC m=+293.575238304" observedRunningTime="2026-05-29 00:52:16.097999378 +0000 UTC m=+294.558202829" watchObservedRunningTime="2026-05-29 00:52:16.100152365 +0000 UTC m=+294.560355814" May 29 00:52:16.111328 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:16.111288 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-t6kgt" podStartSLOduration=1.50135536 podStartE2EDuration="3.111279438s" podCreationTimestamp="2026-05-29 00:52:13 +0000 UTC" firstStartedPulling="2026-05-29 00:52:13.513401354 +0000 UTC m=+291.973604784" lastFinishedPulling="2026-05-29 00:52:15.123325428 +0000 UTC m=+293.583528862" observedRunningTime="2026-05-29 00:52:16.11085819 +0000 UTC m=+294.571061640" watchObservedRunningTime="2026-05-29 00:52:16.111279438 +0000 UTC m=+294.571482890" May 29 00:52:22.045734 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:22.045708 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 00:52:22.046201 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:22.045710 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 00:52:22.062022 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:22.062002 2551 kubelet.go:1628] "Image garbage collection succeeded" May 29 00:52:22.091284 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:22.091186 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-zs5sj" May 29 00:52:47.210763 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.210720 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d"] May 29 00:52:47.218380 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.218358 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.221408 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.221383 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 29 00:52:47.221520 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.221467 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 29 00:52:47.221675 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.221662 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 29 00:52:47.222000 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.221984 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 29 00:52:47.222088 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.222069 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-km7r4\"" May 29 00:52:47.225338 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.225319 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d"] May 29 00:52:47.346681 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.346659 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2425a0be-7ed3-41c3-8052-e8ffa9d27924-webhook-cert\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.346861 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.346700 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2425a0be-7ed3-41c3-8052-e8ffa9d27924-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.346861 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.346743 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljw99\" (UniqueName: \"kubernetes.io/projected/2425a0be-7ed3-41c3-8052-e8ffa9d27924-kube-api-access-ljw99\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.448024 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.447995 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ljw99\" (UniqueName: \"kubernetes.io/projected/2425a0be-7ed3-41c3-8052-e8ffa9d27924-kube-api-access-ljw99\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.448169 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.448051 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2425a0be-7ed3-41c3-8052-e8ffa9d27924-webhook-cert\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.448169 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.448107 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2425a0be-7ed3-41c3-8052-e8ffa9d27924-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.450520 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.450500 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2425a0be-7ed3-41c3-8052-e8ffa9d27924-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.450584 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.450550 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2425a0be-7ed3-41c3-8052-e8ffa9d27924-webhook-cert\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.458935 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.458916 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljw99\" (UniqueName: \"kubernetes.io/projected/2425a0be-7ed3-41c3-8052-e8ffa9d27924-kube-api-access-ljw99\") pod \"opendatahub-operator-controller-manager-d9b8687fd-xhn6d\" (UID: \"2425a0be-7ed3-41c3-8052-e8ffa9d27924\") " pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.530127 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.530106 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:47.663206 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.663179 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d"] May 29 00:52:47.665839 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:52:47.665802 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2425a0be_7ed3_41c3_8052_e8ffa9d27924.slice/crio-2a0534d1dc0c5919904c14363332fd88f57efd918a4feea4ac8eec52091e99c5 WatchSource:0}: Error finding container 2a0534d1dc0c5919904c14363332fd88f57efd918a4feea4ac8eec52091e99c5: Status 404 returned error can't find the container with id 2a0534d1dc0c5919904c14363332fd88f57efd918a4feea4ac8eec52091e99c5 May 29 00:52:47.667455 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:47.667436 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 00:52:48.188584 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:48.188547 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" event={"ID":"2425a0be-7ed3-41c3-8052-e8ffa9d27924","Type":"ContainerStarted","Data":"2a0534d1dc0c5919904c14363332fd88f57efd918a4feea4ac8eec52091e99c5"} May 29 00:52:51.199534 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:51.199494 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" event={"ID":"2425a0be-7ed3-41c3-8052-e8ffa9d27924","Type":"ContainerStarted","Data":"3326abd5db83f6f8691c89fcfe152ea7d41b0480b42823164cf488a9124548f6"} May 29 00:52:51.199907 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:51.199611 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:52:51.223134 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:52:51.223096 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" podStartSLOduration=1.5226387080000001 podStartE2EDuration="4.22308694s" podCreationTimestamp="2026-05-29 00:52:47 +0000 UTC" firstStartedPulling="2026-05-29 00:52:47.667554304 +0000 UTC m=+326.127757733" lastFinishedPulling="2026-05-29 00:52:50.368002533 +0000 UTC m=+328.828205965" observedRunningTime="2026-05-29 00:52:51.22102225 +0000 UTC m=+329.681225700" watchObservedRunningTime="2026-05-29 00:52:51.22308694 +0000 UTC m=+329.683290389" May 29 00:53:02.206305 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:02.206277 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d9b8687fd-xhn6d" May 29 00:53:05.381205 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.381129 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64"] May 29 00:53:05.384276 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.384255 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.388160 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.388141 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5pkd6\"" May 29 00:53:05.388263 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.388159 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 29 00:53:05.388263 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.388141 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 29 00:53:05.393035 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.393016 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64"] May 29 00:53:05.485119 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.485093 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-tls-certs\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.485212 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.485146 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrm5t\" (UniqueName: \"kubernetes.io/projected/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-kube-api-access-wrm5t\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.485262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.485213 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-tmp\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.585989 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.585961 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wrm5t\" (UniqueName: \"kubernetes.io/projected/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-kube-api-access-wrm5t\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.586086 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.586007 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-tmp\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.586086 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.586041 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-tls-certs\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.588282 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.588263 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-tmp\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.588427 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.588411 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-tls-certs\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.593004 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.592986 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrm5t\" (UniqueName: \"kubernetes.io/projected/387c1d2d-b813-4d44-b0cc-35ece0ebd1c3-kube-api-access-wrm5t\") pod \"kube-auth-proxy-7959ccfcc5-4sv64\" (UID: \"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3\") " pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.694827 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.694766 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" May 29 00:53:05.816254 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:05.816231 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64"] May 29 00:53:05.818322 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:53:05.818295 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod387c1d2d_b813_4d44_b0cc_35ece0ebd1c3.slice/crio-53d896c83244d6a07ace6010788b5bfe54bfa286ecd8003a8f185d1c1d879c1b WatchSource:0}: Error finding container 53d896c83244d6a07ace6010788b5bfe54bfa286ecd8003a8f185d1c1d879c1b: Status 404 returned error can't find the container with id 53d896c83244d6a07ace6010788b5bfe54bfa286ecd8003a8f185d1c1d879c1b May 29 00:53:06.247692 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:06.247663 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" event={"ID":"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3","Type":"ContainerStarted","Data":"53d896c83244d6a07ace6010788b5bfe54bfa286ecd8003a8f185d1c1d879c1b"} May 29 00:53:07.988289 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:07.988258 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-bgw8q"] May 29 00:53:07.991569 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:07.991548 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:07.994162 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:07.994140 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" May 29 00:53:07.994250 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:07.994140 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-q549v\"" May 29 00:53:07.997910 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:07.997888 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-bgw8q"] May 29 00:53:08.008399 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:08.008373 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmxf2\" (UniqueName: \"kubernetes.io/projected/20a81a50-d899-4b3a-9fee-db2f1da9ccde-kube-api-access-xmxf2\") pod \"odh-model-controller-858dbf95b8-bgw8q\" (UID: \"20a81a50-d899-4b3a-9fee-db2f1da9ccde\") " pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:08.008518 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:08.008502 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert\") pod \"odh-model-controller-858dbf95b8-bgw8q\" (UID: \"20a81a50-d899-4b3a-9fee-db2f1da9ccde\") " pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:08.109486 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:08.109460 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xmxf2\" (UniqueName: \"kubernetes.io/projected/20a81a50-d899-4b3a-9fee-db2f1da9ccde-kube-api-access-xmxf2\") pod \"odh-model-controller-858dbf95b8-bgw8q\" (UID: \"20a81a50-d899-4b3a-9fee-db2f1da9ccde\") " pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:08.109596 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:08.109523 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert\") pod \"odh-model-controller-858dbf95b8-bgw8q\" (UID: \"20a81a50-d899-4b3a-9fee-db2f1da9ccde\") " pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:08.109670 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:53:08.109650 2551 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 29 00:53:08.109742 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:53:08.109724 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert podName:20a81a50-d899-4b3a-9fee-db2f1da9ccde nodeName:}" failed. No retries permitted until 2026-05-29 00:53:08.609702278 +0000 UTC m=+347.069905720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert") pod "odh-model-controller-858dbf95b8-bgw8q" (UID: "20a81a50-d899-4b3a-9fee-db2f1da9ccde") : secret "odh-model-controller-webhook-cert" not found May 29 00:53:08.118669 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:08.118640 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmxf2\" (UniqueName: \"kubernetes.io/projected/20a81a50-d899-4b3a-9fee-db2f1da9ccde-kube-api-access-xmxf2\") pod \"odh-model-controller-858dbf95b8-bgw8q\" (UID: \"20a81a50-d899-4b3a-9fee-db2f1da9ccde\") " pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:08.612247 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:08.612211 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert\") pod \"odh-model-controller-858dbf95b8-bgw8q\" (UID: \"20a81a50-d899-4b3a-9fee-db2f1da9ccde\") " pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:08.612425 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:53:08.612339 2551 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 29 00:53:08.612425 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:53:08.612408 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert podName:20a81a50-d899-4b3a-9fee-db2f1da9ccde nodeName:}" failed. No retries permitted until 2026-05-29 00:53:09.612389466 +0000 UTC m=+348.072592896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert") pod "odh-model-controller-858dbf95b8-bgw8q" (UID: "20a81a50-d899-4b3a-9fee-db2f1da9ccde") : secret "odh-model-controller-webhook-cert" not found May 29 00:53:09.620602 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:09.620573 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert\") pod \"odh-model-controller-858dbf95b8-bgw8q\" (UID: \"20a81a50-d899-4b3a-9fee-db2f1da9ccde\") " pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:09.622860 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:09.622838 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a81a50-d899-4b3a-9fee-db2f1da9ccde-cert\") pod \"odh-model-controller-858dbf95b8-bgw8q\" (UID: \"20a81a50-d899-4b3a-9fee-db2f1da9ccde\") " pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:09.803060 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:09.803027 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:09.927587 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:09.927535 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-bgw8q"] May 29 00:53:09.929699 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:53:09.929666 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a81a50_d899_4b3a_9fee_db2f1da9ccde.slice/crio-288a1c3cf1d12822b6a5a251f0b0fc562c913f3a98025230b42f5287464afae8 WatchSource:0}: Error finding container 288a1c3cf1d12822b6a5a251f0b0fc562c913f3a98025230b42f5287464afae8: Status 404 returned error can't find the container with id 288a1c3cf1d12822b6a5a251f0b0fc562c913f3a98025230b42f5287464afae8 May 29 00:53:10.261075 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:10.261040 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" event={"ID":"20a81a50-d899-4b3a-9fee-db2f1da9ccde","Type":"ContainerStarted","Data":"288a1c3cf1d12822b6a5a251f0b0fc562c913f3a98025230b42f5287464afae8"} May 29 00:53:10.262311 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:10.262292 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" event={"ID":"387c1d2d-b813-4d44-b0cc-35ece0ebd1c3","Type":"ContainerStarted","Data":"981edb8fad7102c8e1670fe2487161c0f225845152fddb8298c72029a0c124b7"} May 29 00:53:10.279482 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:10.279443 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7959ccfcc5-4sv64" podStartSLOduration=1.883661317 podStartE2EDuration="5.279433186s" podCreationTimestamp="2026-05-29 00:53:05 +0000 UTC" firstStartedPulling="2026-05-29 00:53:05.820522476 +0000 UTC m=+344.280725905" lastFinishedPulling="2026-05-29 00:53:09.216294332 +0000 UTC m=+347.676497774" observedRunningTime="2026-05-29 00:53:10.277121556 +0000 UTC m=+348.737325008" watchObservedRunningTime="2026-05-29 00:53:10.279433186 +0000 UTC m=+348.739636635" May 29 00:53:13.273263 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.273229 2551 generic.go:358] "Generic (PLEG): container finished" podID="20a81a50-d899-4b3a-9fee-db2f1da9ccde" containerID="e29d11ee20adea988bc6e11f5c4755fb5006eeb0e45b5aba5133479d913882ed" exitCode=1 May 29 00:53:13.273691 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.273317 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" event={"ID":"20a81a50-d899-4b3a-9fee-db2f1da9ccde","Type":"ContainerDied","Data":"e29d11ee20adea988bc6e11f5c4755fb5006eeb0e45b5aba5133479d913882ed"} May 29 00:53:13.273691 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.273508 2551 scope.go:117] "RemoveContainer" containerID="e29d11ee20adea988bc6e11f5c4755fb5006eeb0e45b5aba5133479d913882ed" May 29 00:53:13.396368 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.396344 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-8srdn"] May 29 00:53:13.399456 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.399443 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:13.402206 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.402176 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" May 29 00:53:13.402327 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.402279 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-g8mrz\"" May 29 00:53:13.408881 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.408859 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-8srdn"] May 29 00:53:13.452678 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.452648 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhzbx\" (UniqueName: \"kubernetes.io/projected/efba5428-b3ed-4d0c-9b29-0c4929f74e65-kube-api-access-jhzbx\") pod \"kserve-controller-manager-856948b99f-8srdn\" (UID: \"efba5428-b3ed-4d0c-9b29-0c4929f74e65\") " pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:13.452840 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.452701 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efba5428-b3ed-4d0c-9b29-0c4929f74e65-cert\") pod \"kserve-controller-manager-856948b99f-8srdn\" (UID: \"efba5428-b3ed-4d0c-9b29-0c4929f74e65\") " pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:13.553999 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.553978 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jhzbx\" (UniqueName: \"kubernetes.io/projected/efba5428-b3ed-4d0c-9b29-0c4929f74e65-kube-api-access-jhzbx\") pod \"kserve-controller-manager-856948b99f-8srdn\" (UID: \"efba5428-b3ed-4d0c-9b29-0c4929f74e65\") " pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:13.554113 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.554039 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efba5428-b3ed-4d0c-9b29-0c4929f74e65-cert\") pod \"kserve-controller-manager-856948b99f-8srdn\" (UID: \"efba5428-b3ed-4d0c-9b29-0c4929f74e65\") " pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:13.554174 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:53:13.554141 2551 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 29 00:53:13.554225 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:53:13.554200 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efba5428-b3ed-4d0c-9b29-0c4929f74e65-cert podName:efba5428-b3ed-4d0c-9b29-0c4929f74e65 nodeName:}" failed. No retries permitted until 2026-05-29 00:53:14.054177777 +0000 UTC m=+352.514381205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/efba5428-b3ed-4d0c-9b29-0c4929f74e65-cert") pod "kserve-controller-manager-856948b99f-8srdn" (UID: "efba5428-b3ed-4d0c-9b29-0c4929f74e65") : secret "kserve-webhook-server-cert" not found May 29 00:53:13.564726 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:13.564698 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhzbx\" (UniqueName: \"kubernetes.io/projected/efba5428-b3ed-4d0c-9b29-0c4929f74e65-kube-api-access-jhzbx\") pod \"kserve-controller-manager-856948b99f-8srdn\" (UID: \"efba5428-b3ed-4d0c-9b29-0c4929f74e65\") " pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:14.058530 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:14.058493 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efba5428-b3ed-4d0c-9b29-0c4929f74e65-cert\") pod \"kserve-controller-manager-856948b99f-8srdn\" (UID: \"efba5428-b3ed-4d0c-9b29-0c4929f74e65\") " pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:14.060822 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:14.060794 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efba5428-b3ed-4d0c-9b29-0c4929f74e65-cert\") pod \"kserve-controller-manager-856948b99f-8srdn\" (UID: \"efba5428-b3ed-4d0c-9b29-0c4929f74e65\") " pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:14.278556 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:14.278515 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" event={"ID":"20a81a50-d899-4b3a-9fee-db2f1da9ccde","Type":"ContainerStarted","Data":"1dfd151b2d439077b3d6315de17accb93c13f5a4835604440251d62c12bb6399"} May 29 00:53:14.279006 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:14.278685 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:14.296555 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:14.296512 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" podStartSLOduration=3.719771152 podStartE2EDuration="7.29649829s" podCreationTimestamp="2026-05-29 00:53:07 +0000 UTC" firstStartedPulling="2026-05-29 00:53:09.931005643 +0000 UTC m=+348.391209071" lastFinishedPulling="2026-05-29 00:53:13.507732766 +0000 UTC m=+351.967936209" observedRunningTime="2026-05-29 00:53:14.294968988 +0000 UTC m=+352.755172440" watchObservedRunningTime="2026-05-29 00:53:14.29649829 +0000 UTC m=+352.756701741" May 29 00:53:14.311363 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:14.311311 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:14.436422 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:14.436397 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-8srdn"] May 29 00:53:14.438887 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:53:14.438856 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefba5428_b3ed_4d0c_9b29_0c4929f74e65.slice/crio-4af05a36b9284d5f97db87dc1bf2385fbe574b317394228ca523a92e4bf8b455 WatchSource:0}: Error finding container 4af05a36b9284d5f97db87dc1bf2385fbe574b317394228ca523a92e4bf8b455: Status 404 returned error can't find the container with id 4af05a36b9284d5f97db87dc1bf2385fbe574b317394228ca523a92e4bf8b455 May 29 00:53:15.283583 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:15.283544 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" event={"ID":"efba5428-b3ed-4d0c-9b29-0c4929f74e65","Type":"ContainerStarted","Data":"4af05a36b9284d5f97db87dc1bf2385fbe574b317394228ca523a92e4bf8b455"} May 29 00:53:18.293834 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:18.293792 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" event={"ID":"efba5428-b3ed-4d0c-9b29-0c4929f74e65","Type":"ContainerStarted","Data":"b203fdb9c89bcde9661b70e61fc70661aa5f598ee053b48c22eedf9fba1cabff"} May 29 00:53:18.294218 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:18.293958 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:53:18.308947 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:18.308899 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" podStartSLOduration=2.046970274 podStartE2EDuration="5.308885001s" podCreationTimestamp="2026-05-29 00:53:13 +0000 UTC" firstStartedPulling="2026-05-29 00:53:14.440650397 +0000 UTC m=+352.900853837" lastFinishedPulling="2026-05-29 00:53:17.702565133 +0000 UTC m=+356.162768564" observedRunningTime="2026-05-29 00:53:18.308398289 +0000 UTC m=+356.768601738" watchObservedRunningTime="2026-05-29 00:53:18.308885001 +0000 UTC m=+356.769088452" May 29 00:53:22.084314 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.084285 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx"] May 29 00:53:22.087622 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.087605 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:22.090958 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.090938 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-mlvxl\"" May 29 00:53:22.091103 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.091074 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" May 29 00:53:22.091188 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.091174 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" May 29 00:53:22.103426 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.103397 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx"] May 29 00:53:22.118798 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.118774 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55x88\" (UniqueName: \"kubernetes.io/projected/96fce155-f8e8-447e-9f47-75c0f9f6c192-kube-api-access-55x88\") pod \"servicemesh-operator3-55f49c5f94-qcfhx\" (UID: \"96fce155-f8e8-447e-9f47-75c0f9f6c192\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:22.118939 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.118918 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/96fce155-f8e8-447e-9f47-75c0f9f6c192-operator-config\") pod \"servicemesh-operator3-55f49c5f94-qcfhx\" (UID: \"96fce155-f8e8-447e-9f47-75c0f9f6c192\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:22.219682 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.219663 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-55x88\" (UniqueName: \"kubernetes.io/projected/96fce155-f8e8-447e-9f47-75c0f9f6c192-kube-api-access-55x88\") pod \"servicemesh-operator3-55f49c5f94-qcfhx\" (UID: \"96fce155-f8e8-447e-9f47-75c0f9f6c192\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:22.219812 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.219697 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/96fce155-f8e8-447e-9f47-75c0f9f6c192-operator-config\") pod \"servicemesh-operator3-55f49c5f94-qcfhx\" (UID: \"96fce155-f8e8-447e-9f47-75c0f9f6c192\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:22.222016 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.221992 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/96fce155-f8e8-447e-9f47-75c0f9f6c192-operator-config\") pod \"servicemesh-operator3-55f49c5f94-qcfhx\" (UID: \"96fce155-f8e8-447e-9f47-75c0f9f6c192\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:22.227429 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.227397 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55x88\" (UniqueName: \"kubernetes.io/projected/96fce155-f8e8-447e-9f47-75c0f9f6c192-kube-api-access-55x88\") pod \"servicemesh-operator3-55f49c5f94-qcfhx\" (UID: \"96fce155-f8e8-447e-9f47-75c0f9f6c192\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:22.397054 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.396978 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:22.528315 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:22.528289 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx"] May 29 00:53:22.530357 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:53:22.530318 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fce155_f8e8_447e_9f47_75c0f9f6c192.slice/crio-057e0d74bc2297f03451ead9ca12464687a31c6e247eb5ea540faf14ed2b311e WatchSource:0}: Error finding container 057e0d74bc2297f03451ead9ca12464687a31c6e247eb5ea540faf14ed2b311e: Status 404 returned error can't find the container with id 057e0d74bc2297f03451ead9ca12464687a31c6e247eb5ea540faf14ed2b311e May 29 00:53:23.316467 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:23.316431 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" event={"ID":"96fce155-f8e8-447e-9f47-75c0f9f6c192","Type":"ContainerStarted","Data":"057e0d74bc2297f03451ead9ca12464687a31c6e247eb5ea540faf14ed2b311e"} May 29 00:53:25.286370 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:25.286342 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-bgw8q" May 29 00:53:25.325952 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:25.325919 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" event={"ID":"96fce155-f8e8-447e-9f47-75c0f9f6c192","Type":"ContainerStarted","Data":"ae0b377af50b143b49fad9a54aa3a3732703d1e1e0fdd485f67e125b7e9d57d8"} May 29 00:53:25.326120 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:25.326021 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:25.344447 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:25.344398 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" podStartSLOduration=1.191015336 podStartE2EDuration="3.344381085s" podCreationTimestamp="2026-05-29 00:53:22 +0000 UTC" firstStartedPulling="2026-05-29 00:53:22.532806131 +0000 UTC m=+360.993009560" lastFinishedPulling="2026-05-29 00:53:24.686171872 +0000 UTC m=+363.146375309" observedRunningTime="2026-05-29 00:53:25.342133815 +0000 UTC m=+363.802337264" watchObservedRunningTime="2026-05-29 00:53:25.344381085 +0000 UTC m=+363.804584535" May 29 00:53:32.849098 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.849063 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j"] May 29 00:53:32.853111 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.853085 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:32.855617 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.855577 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" May 29 00:53:32.855846 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.855667 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5c9rw\"" May 29 00:53:32.855846 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.855724 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 29 00:53:32.856000 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.855965 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" May 29 00:53:32.856334 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.856312 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" May 29 00:53:32.860179 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.860154 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j"] May 29 00:53:32.898528 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.898434 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a585311-2208-477b-970f-086368c6e6aa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:32.898528 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.898492 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-cacerts\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:32.898528 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.898541 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7czjf\" (UniqueName: \"kubernetes.io/projected/7a585311-2208-477b-970f-086368c6e6aa-kube-api-access-7czjf\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:32.898857 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.898583 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:32.898857 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.898667 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a585311-2208-477b-970f-086368c6e6aa-istio-token\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:32.898857 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.898723 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a585311-2208-477b-970f-086368c6e6aa-local-certs\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:32.898857 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.898773 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.000024 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:32.999999 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7czjf\" (UniqueName: \"kubernetes.io/projected/7a585311-2208-477b-970f-086368c6e6aa-kube-api-access-7czjf\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.000158 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.000041 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.000158 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.000147 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a585311-2208-477b-970f-086368c6e6aa-istio-token\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.000270 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.000194 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a585311-2208-477b-970f-086368c6e6aa-local-certs\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.000270 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.000223 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.000270 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.000259 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a585311-2208-477b-970f-086368c6e6aa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.000410 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.000310 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-cacerts\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.000955 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.000930 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7a585311-2208-477b-970f-086368c6e6aa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.002531 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.002497 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a585311-2208-477b-970f-086368c6e6aa-local-certs\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.002791 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.002769 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-cacerts\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.002880 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.002861 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.003149 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.003129 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7a585311-2208-477b-970f-086368c6e6aa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.007709 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.007689 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a585311-2208-477b-970f-086368c6e6aa-istio-token\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.007987 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.007969 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7czjf\" (UniqueName: \"kubernetes.io/projected/7a585311-2208-477b-970f-086368c6e6aa-kube-api-access-7czjf\") pod \"istiod-openshift-gateway-798958bb55-xpx5j\" (UID: \"7a585311-2208-477b-970f-086368c6e6aa\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.165403 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.165330 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:33.299492 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.299460 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j"] May 29 00:53:33.301136 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:53:33.301098 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a585311_2208_477b_970f_086368c6e6aa.slice/crio-02991b8143e907b0e4c549d269f60e93f720f9589eb650c7965ba0b8b692d0c0 WatchSource:0}: Error finding container 02991b8143e907b0e4c549d269f60e93f720f9589eb650c7965ba0b8b692d0c0: Status 404 returned error can't find the container with id 02991b8143e907b0e4c549d269f60e93f720f9589eb650c7965ba0b8b692d0c0 May 29 00:53:33.350716 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:33.350685 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" event={"ID":"7a585311-2208-477b-970f-086368c6e6aa","Type":"ContainerStarted","Data":"02991b8143e907b0e4c549d269f60e93f720f9589eb650c7965ba0b8b692d0c0"} May 29 00:53:36.081810 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:36.081770 2551 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 29 00:53:36.082091 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:36.081845 2551 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 29 00:53:36.331034 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:36.330964 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-qcfhx" May 29 00:53:36.364597 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:36.364564 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" event={"ID":"7a585311-2208-477b-970f-086368c6e6aa","Type":"ContainerStarted","Data":"a481f56ae17e9aa5056892a794e7af96e1c834f6df814d69be41bacb307c8881"} May 29 00:53:36.364793 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:36.364717 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:36.383356 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:36.383290 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" podStartSLOduration=1.605137273 podStartE2EDuration="4.383273912s" podCreationTimestamp="2026-05-29 00:53:32 +0000 UTC" firstStartedPulling="2026-05-29 00:53:33.303393674 +0000 UTC m=+371.763597105" lastFinishedPulling="2026-05-29 00:53:36.081530316 +0000 UTC m=+374.541733744" observedRunningTime="2026-05-29 00:53:36.382134415 +0000 UTC m=+374.842337865" watchObservedRunningTime="2026-05-29 00:53:36.383273912 +0000 UTC m=+374.843477363" May 29 00:53:37.370649 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:37.370621 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xpx5j" May 29 00:53:49.301417 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:53:49.301386 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-8srdn" May 29 00:54:39.220440 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.220357 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj"] May 29 00:54:39.223825 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.223802 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:39.228053 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.228033 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 29 00:54:39.228490 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.228471 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f6qxk\"" May 29 00:54:39.231007 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.230984 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 29 00:54:39.243318 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.243293 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj"] May 29 00:54:39.312497 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.312462 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khfzw\" (UniqueName: \"kubernetes.io/projected/33e50f6f-2f59-4645-aaa2-23332b36591f-kube-api-access-khfzw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" (UID: \"33e50f6f-2f59-4645-aaa2-23332b36591f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:39.312607 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.312519 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e50f6f-2f59-4645-aaa2-23332b36591f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" (UID: \"33e50f6f-2f59-4645-aaa2-23332b36591f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:39.413617 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.413590 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-khfzw\" (UniqueName: \"kubernetes.io/projected/33e50f6f-2f59-4645-aaa2-23332b36591f-kube-api-access-khfzw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" (UID: \"33e50f6f-2f59-4645-aaa2-23332b36591f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:39.413715 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.413632 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e50f6f-2f59-4645-aaa2-23332b36591f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" (UID: \"33e50f6f-2f59-4645-aaa2-23332b36591f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:39.414000 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.413984 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e50f6f-2f59-4645-aaa2-23332b36591f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" (UID: \"33e50f6f-2f59-4645-aaa2-23332b36591f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:39.424944 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.424920 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khfzw\" (UniqueName: \"kubernetes.io/projected/33e50f6f-2f59-4645-aaa2-23332b36591f-kube-api-access-khfzw\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" (UID: \"33e50f6f-2f59-4645-aaa2-23332b36591f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:39.534088 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.534026 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:39.673988 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:39.673965 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj"] May 29 00:54:39.676276 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:54:39.676248 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e50f6f_2f59_4645_aaa2_23332b36591f.slice/crio-e1250970dcaab0d2b3340bc2a59daf83e29752ba2c0ef250abdfe0c93803bec2 WatchSource:0}: Error finding container e1250970dcaab0d2b3340bc2a59daf83e29752ba2c0ef250abdfe0c93803bec2: Status 404 returned error can't find the container with id e1250970dcaab0d2b3340bc2a59daf83e29752ba2c0ef250abdfe0c93803bec2 May 29 00:54:40.577499 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:40.577452 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" event={"ID":"33e50f6f-2f59-4645-aaa2-23332b36591f","Type":"ContainerStarted","Data":"e1250970dcaab0d2b3340bc2a59daf83e29752ba2c0ef250abdfe0c93803bec2"} May 29 00:54:41.757109 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:41.757077 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-rg7bm"] May 29 00:54:41.760606 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:41.760581 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" May 29 00:54:41.763269 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:41.763249 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-m5kbj\"" May 29 00:54:41.767959 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:41.767936 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-rg7bm"] May 29 00:54:41.835375 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:41.835347 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj257\" (UniqueName: \"kubernetes.io/projected/1cdb5df8-9337-4ee0-82f6-364c86299e0c-kube-api-access-kj257\") pod \"authorino-operator-657f44b778-rg7bm\" (UID: \"1cdb5df8-9337-4ee0-82f6-364c86299e0c\") " pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" May 29 00:54:41.936659 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:41.936625 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kj257\" (UniqueName: \"kubernetes.io/projected/1cdb5df8-9337-4ee0-82f6-364c86299e0c-kube-api-access-kj257\") pod \"authorino-operator-657f44b778-rg7bm\" (UID: \"1cdb5df8-9337-4ee0-82f6-364c86299e0c\") " pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" May 29 00:54:41.944281 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:41.944261 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj257\" (UniqueName: \"kubernetes.io/projected/1cdb5df8-9337-4ee0-82f6-364c86299e0c-kube-api-access-kj257\") pod \"authorino-operator-657f44b778-rg7bm\" (UID: \"1cdb5df8-9337-4ee0-82f6-364c86299e0c\") " pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" May 29 00:54:42.073853 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:42.073797 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" May 29 00:54:42.209609 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:42.209568 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-rg7bm"] May 29 00:54:42.213120 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:54:42.213078 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cdb5df8_9337_4ee0_82f6_364c86299e0c.slice/crio-6fe742ef82727eb0de80e5b0b6a225ce1ebdffcf55223ef6d15791679dcb1ef1 WatchSource:0}: Error finding container 6fe742ef82727eb0de80e5b0b6a225ce1ebdffcf55223ef6d15791679dcb1ef1: Status 404 returned error can't find the container with id 6fe742ef82727eb0de80e5b0b6a225ce1ebdffcf55223ef6d15791679dcb1ef1 May 29 00:54:42.585282 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:42.585255 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" event={"ID":"1cdb5df8-9337-4ee0-82f6-364c86299e0c","Type":"ContainerStarted","Data":"6fe742ef82727eb0de80e5b0b6a225ce1ebdffcf55223ef6d15791679dcb1ef1"} May 29 00:54:44.594419 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:44.594381 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" event={"ID":"1cdb5df8-9337-4ee0-82f6-364c86299e0c","Type":"ContainerStarted","Data":"c6b7977f687968aad132c50fe001eaa27f1fa50f2284156c2a5a9a3d149bced0"} May 29 00:54:44.595056 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:44.594486 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" May 29 00:54:44.611591 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:44.611542 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" podStartSLOduration=1.727657936 podStartE2EDuration="3.611528447s" podCreationTimestamp="2026-05-29 00:54:41 +0000 UTC" firstStartedPulling="2026-05-29 00:54:42.215702315 +0000 UTC m=+440.675905743" lastFinishedPulling="2026-05-29 00:54:44.099572822 +0000 UTC m=+442.559776254" observedRunningTime="2026-05-29 00:54:44.609323983 +0000 UTC m=+443.069527434" watchObservedRunningTime="2026-05-29 00:54:44.611528447 +0000 UTC m=+443.071731897" May 29 00:54:46.604454 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:46.604418 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" event={"ID":"33e50f6f-2f59-4645-aaa2-23332b36591f","Type":"ContainerStarted","Data":"11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a"} May 29 00:54:46.604869 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:46.604548 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:46.625340 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:46.625294 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" podStartSLOduration=1.7119951389999999 podStartE2EDuration="7.625280499s" podCreationTimestamp="2026-05-29 00:54:39 +0000 UTC" firstStartedPulling="2026-05-29 00:54:39.681149506 +0000 UTC m=+438.141352940" lastFinishedPulling="2026-05-29 00:54:45.594434867 +0000 UTC m=+444.054638300" observedRunningTime="2026-05-29 00:54:46.623152245 +0000 UTC m=+445.083355698" watchObservedRunningTime="2026-05-29 00:54:46.625280499 +0000 UTC m=+445.085483950" May 29 00:54:55.602502 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:55.602467 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-rg7bm" May 29 00:54:57.609720 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:57.609692 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:59.314816 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.314774 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj"] May 29 00:54:59.315203 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.314996 2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" containerName="manager" containerID="cri-o://11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a" gracePeriod=2 May 29 00:54:59.322443 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.322388 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj"] May 29 00:54:59.336585 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.336560 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw"] May 29 00:54:59.336923 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.336910 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" containerName="manager" May 29 00:54:59.336976 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.336924 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" containerName="manager" May 29 00:54:59.337012 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.336981 2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" containerName="manager" May 29 00:54:59.340087 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.340071 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:54:59.351396 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.351375 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw"] May 29 00:54:59.369522 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.369493 2551 status_manager.go:895] "Failed to get status for pod" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" is forbidden: User \"system:node:ip-10-0-141-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-236.ec2.internal' and this object" May 29 00:54:59.472670 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.472633 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-468b4\" (UniqueName: \"kubernetes.io/projected/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-kube-api-access-468b4\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-drsnw\" (UID: \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:54:59.472670 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.472673 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-drsnw\" (UID: \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:54:59.557593 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.557571 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:59.561130 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.561106 2551 status_manager.go:895] "Failed to get status for pod" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" is forbidden: User \"system:node:ip-10-0-141-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-236.ec2.internal' and this object" May 29 00:54:59.573566 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.573512 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-468b4\" (UniqueName: \"kubernetes.io/projected/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-kube-api-access-468b4\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-drsnw\" (UID: \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:54:59.573566 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.573544 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-drsnw\" (UID: \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:54:59.573861 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.573845 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-drsnw\" (UID: \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:54:59.582081 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.582063 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-468b4\" (UniqueName: \"kubernetes.io/projected/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-kube-api-access-468b4\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-drsnw\" (UID: \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:54:59.658442 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.658414 2551 generic.go:358] "Generic (PLEG): container finished" podID="33e50f6f-2f59-4645-aaa2-23332b36591f" containerID="11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a" exitCode=0 May 29 00:54:59.658526 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.658457 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" May 29 00:54:59.658526 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.658480 2551 scope.go:117] "RemoveContainer" containerID="11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a" May 29 00:54:59.661451 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.661431 2551 status_manager.go:895] "Failed to get status for pod" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" is forbidden: User \"system:node:ip-10-0-141-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-236.ec2.internal' and this object" May 29 00:54:59.665922 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.665904 2551 scope.go:117] "RemoveContainer" containerID="11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a" May 29 00:54:59.666183 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:54:59.666153 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a\": container with ID starting with 11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a not found: ID does not exist" containerID="11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a" May 29 00:54:59.666229 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.666190 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a"} err="failed to get container status \"11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a\": rpc error: code = NotFound desc = could not find container \"11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a\": container with ID starting with 11437485af3d618d53e89d375ed1feca6a4981cd719f6fac12896ddc8ec54c4a not found: ID does not exist" May 29 00:54:59.674464 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.674448 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khfzw\" (UniqueName: \"kubernetes.io/projected/33e50f6f-2f59-4645-aaa2-23332b36591f-kube-api-access-khfzw\") pod \"33e50f6f-2f59-4645-aaa2-23332b36591f\" (UID: \"33e50f6f-2f59-4645-aaa2-23332b36591f\") " May 29 00:54:59.674507 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.674500 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e50f6f-2f59-4645-aaa2-23332b36591f-extensions-socket-volume\") pod \"33e50f6f-2f59-4645-aaa2-23332b36591f\" (UID: \"33e50f6f-2f59-4645-aaa2-23332b36591f\") " May 29 00:54:59.674972 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.674949 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e50f6f-2f59-4645-aaa2-23332b36591f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "33e50f6f-2f59-4645-aaa2-23332b36591f" (UID: "33e50f6f-2f59-4645-aaa2-23332b36591f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 29 00:54:59.676430 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.676406 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e50f6f-2f59-4645-aaa2-23332b36591f-kube-api-access-khfzw" (OuterVolumeSpecName: "kube-api-access-khfzw") pod "33e50f6f-2f59-4645-aaa2-23332b36591f" (UID: "33e50f6f-2f59-4645-aaa2-23332b36591f"). InnerVolumeSpecName "kube-api-access-khfzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 00:54:59.688618 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.688594 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:54:59.775546 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.775514 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khfzw\" (UniqueName: \"kubernetes.io/projected/33e50f6f-2f59-4645-aaa2-23332b36591f-kube-api-access-khfzw\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:54:59.775645 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.775555 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e50f6f-2f59-4645-aaa2-23332b36591f-extensions-socket-volume\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:54:59.824886 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.824817 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw"] May 29 00:54:59.830249 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:54:59.830225 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae791456_bbd0_4b2e_ad76_4c3fdde44a59.slice/crio-481533e4913f209351f0801d84848126874e9aac0d688524bfe9ecabf6c7aac9 WatchSource:0}: Error finding container 481533e4913f209351f0801d84848126874e9aac0d688524bfe9ecabf6c7aac9: Status 404 returned error can't find the container with id 481533e4913f209351f0801d84848126874e9aac0d688524bfe9ecabf6c7aac9 May 29 00:54:59.971234 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:54:59.971207 2551 status_manager.go:895] "Failed to get status for pod" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5p6fj" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-5p6fj\" is forbidden: User \"system:node:ip-10-0-141-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-236.ec2.internal' and this object" May 29 00:55:00.141935 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:00.141857 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e50f6f-2f59-4645-aaa2-23332b36591f" path="/var/lib/kubelet/pods/33e50f6f-2f59-4645-aaa2-23332b36591f/volumes" May 29 00:55:00.663993 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:00.663956 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" event={"ID":"ae791456-bbd0-4b2e-ad76-4c3fdde44a59","Type":"ContainerStarted","Data":"8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec"} May 29 00:55:00.663993 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:00.663994 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" event={"ID":"ae791456-bbd0-4b2e-ad76-4c3fdde44a59","Type":"ContainerStarted","Data":"481533e4913f209351f0801d84848126874e9aac0d688524bfe9ecabf6c7aac9"} May 29 00:55:00.664488 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:00.664103 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:55:00.686346 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:00.686300 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" podStartSLOduration=1.686285522 podStartE2EDuration="1.686285522s" podCreationTimestamp="2026-05-29 00:54:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 00:55:00.685678124 +0000 UTC m=+459.145881568" watchObservedRunningTime="2026-05-29 00:55:00.686285522 +0000 UTC m=+459.146488970" May 29 00:55:01.115243 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.115214 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bd5d89bf9-mxxbv"] May 29 00:55:01.118744 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.118722 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.130804 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.130777 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bd5d89bf9-mxxbv"] May 29 00:55:01.186768 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.186730 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-console-config\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.186878 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.186770 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-oauth-serving-cert\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.186878 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.186824 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn5rj\" (UniqueName: \"kubernetes.io/projected/6f082769-70a1-4fb6-9283-bfabdc28318a-kube-api-access-pn5rj\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.186950 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.186893 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f082769-70a1-4fb6-9283-bfabdc28318a-console-oauth-config\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.186950 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.186925 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-service-ca\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.187036 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.186948 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-trusted-ca-bundle\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.187036 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.186974 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f082769-70a1-4fb6-9283-bfabdc28318a-console-serving-cert\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.287993 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.287968 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f082769-70a1-4fb6-9283-bfabdc28318a-console-serving-cert\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288133 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288008 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-console-config\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288133 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288027 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-oauth-serving-cert\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288133 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288045 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pn5rj\" (UniqueName: \"kubernetes.io/projected/6f082769-70a1-4fb6-9283-bfabdc28318a-kube-api-access-pn5rj\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288133 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288093 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f082769-70a1-4fb6-9283-bfabdc28318a-console-oauth-config\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288133 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288114 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-service-ca\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288380 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288138 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-trusted-ca-bundle\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288869 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288842 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-oauth-serving-cert\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288974 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288867 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-console-config\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288974 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288913 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-service-ca\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.288974 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.288915 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f082769-70a1-4fb6-9283-bfabdc28318a-trusted-ca-bundle\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.290591 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.290565 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f082769-70a1-4fb6-9283-bfabdc28318a-console-oauth-config\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.290697 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.290630 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f082769-70a1-4fb6-9283-bfabdc28318a-console-serving-cert\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.296619 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.296601 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn5rj\" (UniqueName: \"kubernetes.io/projected/6f082769-70a1-4fb6-9283-bfabdc28318a-kube-api-access-pn5rj\") pod \"console-6bd5d89bf9-mxxbv\" (UID: \"6f082769-70a1-4fb6-9283-bfabdc28318a\") " pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.432951 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.432892 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:01.563006 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.562977 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bd5d89bf9-mxxbv"] May 29 00:55:01.564946 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:55:01.564919 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f082769_70a1_4fb6_9283_bfabdc28318a.slice/crio-bf91d560b5f302bb7b9f8b4bc8694cd117d1dfee15ff937642f6e27c0911a21a WatchSource:0}: Error finding container bf91d560b5f302bb7b9f8b4bc8694cd117d1dfee15ff937642f6e27c0911a21a: Status 404 returned error can't find the container with id bf91d560b5f302bb7b9f8b4bc8694cd117d1dfee15ff937642f6e27c0911a21a May 29 00:55:01.672915 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.672865 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd5d89bf9-mxxbv" event={"ID":"6f082769-70a1-4fb6-9283-bfabdc28318a","Type":"ContainerStarted","Data":"25275d0adb8a5385f1a533dca8b3eaa1a8ed7057be929e4ea78b2b4d5b025a45"} May 29 00:55:01.672915 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.672926 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bd5d89bf9-mxxbv" event={"ID":"6f082769-70a1-4fb6-9283-bfabdc28318a","Type":"ContainerStarted","Data":"bf91d560b5f302bb7b9f8b4bc8694cd117d1dfee15ff937642f6e27c0911a21a"} May 29 00:55:01.691792 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:01.691669 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6bd5d89bf9-mxxbv" podStartSLOduration=0.691656017 podStartE2EDuration="691.656017ms" podCreationTimestamp="2026-05-29 00:55:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 00:55:01.690042045 +0000 UTC m=+460.150245509" watchObservedRunningTime="2026-05-29 00:55:01.691656017 +0000 UTC m=+460.151859466" May 29 00:55:11.434639 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:11.434564 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:11.435151 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:11.434940 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:11.439590 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:11.439565 2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:11.674853 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:11.674829 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:55:11.711240 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:11.711174 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bd5d89bf9-mxxbv" May 29 00:55:11.763679 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:11.763642 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66f9f94d8f-kbm94"] May 29 00:55:26.666479 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.666432 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw"] May 29 00:55:26.667050 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.666780 2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" podUID="ae791456-bbd0-4b2e-ad76-4c3fdde44a59" containerName="manager" containerID="cri-o://8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec" gracePeriod=10 May 29 00:55:26.885049 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.885021 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c"] May 29 00:55:26.887356 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.887341 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:26.891409 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.891379 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-scm6c\" (UID: \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:26.891515 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.891474 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxt4b\" (UniqueName: \"kubernetes.io/projected/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-kube-api-access-wxt4b\") pod \"kuadrant-operator-controller-manager-55c7f4c975-scm6c\" (UID: \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:26.898038 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.898012 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c"] May 29 00:55:26.902850 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.902831 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:55:26.992881 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.992850 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxt4b\" (UniqueName: \"kubernetes.io/projected/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-kube-api-access-wxt4b\") pod \"kuadrant-operator-controller-manager-55c7f4c975-scm6c\" (UID: \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:26.993017 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.992906 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-scm6c\" (UID: \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:26.993263 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:26.993245 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-scm6c\" (UID: \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:27.000876 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.000853 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxt4b\" (UniqueName: \"kubernetes.io/projected/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-kube-api-access-wxt4b\") pod \"kuadrant-operator-controller-manager-55c7f4c975-scm6c\" (UID: \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:27.093958 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.093936 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-extensions-socket-volume\") pod \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\" (UID: \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\") " May 29 00:55:27.094067 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.093978 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-468b4\" (UniqueName: \"kubernetes.io/projected/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-kube-api-access-468b4\") pod \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\" (UID: \"ae791456-bbd0-4b2e-ad76-4c3fdde44a59\") " May 29 00:55:27.094319 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.094285 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ae791456-bbd0-4b2e-ad76-4c3fdde44a59" (UID: "ae791456-bbd0-4b2e-ad76-4c3fdde44a59"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 29 00:55:27.095957 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.095935 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-kube-api-access-468b4" (OuterVolumeSpecName: "kube-api-access-468b4") pod "ae791456-bbd0-4b2e-ad76-4c3fdde44a59" (UID: "ae791456-bbd0-4b2e-ad76-4c3fdde44a59"). InnerVolumeSpecName "kube-api-access-468b4". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 00:55:27.194621 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.194587 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-extensions-socket-volume\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:27.194621 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.194613 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-468b4\" (UniqueName: \"kubernetes.io/projected/ae791456-bbd0-4b2e-ad76-4c3fdde44a59-kube-api-access-468b4\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:27.210139 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.210118 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:27.337351 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.337313 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c"] May 29 00:55:27.340939 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:55:27.340912 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9e73e4_3610_49d3_9b61_04d81bf88fb0.slice/crio-900aab3ee092a8f4d07ac581659379f15bc5fdb6aeadc868d32fb3356375236e WatchSource:0}: Error finding container 900aab3ee092a8f4d07ac581659379f15bc5fdb6aeadc868d32fb3356375236e: Status 404 returned error can't find the container with id 900aab3ee092a8f4d07ac581659379f15bc5fdb6aeadc868d32fb3356375236e May 29 00:55:27.774409 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.774378 2551 generic.go:358] "Generic (PLEG): container finished" podID="ae791456-bbd0-4b2e-ad76-4c3fdde44a59" containerID="8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec" exitCode=0 May 29 00:55:27.774838 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.774463 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" May 29 00:55:27.774838 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.774467 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" event={"ID":"ae791456-bbd0-4b2e-ad76-4c3fdde44a59","Type":"ContainerDied","Data":"8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec"} May 29 00:55:27.774838 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.774525 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw" event={"ID":"ae791456-bbd0-4b2e-ad76-4c3fdde44a59","Type":"ContainerDied","Data":"481533e4913f209351f0801d84848126874e9aac0d688524bfe9ecabf6c7aac9"} May 29 00:55:27.774838 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.774549 2551 scope.go:117] "RemoveContainer" containerID="8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec" May 29 00:55:27.776020 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.776000 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" event={"ID":"fa9e73e4-3610-49d3-9b61-04d81bf88fb0","Type":"ContainerStarted","Data":"f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47"} May 29 00:55:27.776124 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.776024 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" event={"ID":"fa9e73e4-3610-49d3-9b61-04d81bf88fb0","Type":"ContainerStarted","Data":"900aab3ee092a8f4d07ac581659379f15bc5fdb6aeadc868d32fb3356375236e"} May 29 00:55:27.776179 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.776161 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:55:27.782870 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.782558 2551 scope.go:117] "RemoveContainer" containerID="8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec" May 29 00:55:27.782870 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:55:27.782836 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec\": container with ID starting with 8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec not found: ID does not exist" containerID="8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec" May 29 00:55:27.782965 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.782866 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec"} err="failed to get container status \"8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec\": rpc error: code = NotFound desc = could not find container \"8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec\": container with ID starting with 8a14ce97fe12f4e9c0c26ea66ab15d3a56e42e954c3df55442d657dfac4983ec not found: ID does not exist" May 29 00:55:27.796760 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.796716 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" podStartSLOduration=1.79670498 podStartE2EDuration="1.79670498s" podCreationTimestamp="2026-05-29 00:55:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 00:55:27.795365672 +0000 UTC m=+486.255569133" watchObservedRunningTime="2026-05-29 00:55:27.79670498 +0000 UTC m=+486.256908433" May 29 00:55:27.818497 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.818475 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw"] May 29 00:55:27.826152 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:27.826135 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-drsnw"] May 29 00:55:28.143072 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:28.143000 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae791456-bbd0-4b2e-ad76-4c3fdde44a59" path="/var/lib/kubelet/pods/ae791456-bbd0-4b2e-ad76-4c3fdde44a59/volumes" May 29 00:55:36.784147 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:36.784106 2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-66f9f94d8f-kbm94" podUID="a02d348b-9e24-4ea7-b372-1e9d19f7507b" containerName="console" containerID="cri-o://ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99" gracePeriod=15 May 29 00:55:37.042015 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.041966 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66f9f94d8f-kbm94_a02d348b-9e24-4ea7-b372-1e9d19f7507b/console/0.log" May 29 00:55:37.042120 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.042023 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:55:37.061744 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.061717 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-service-ca\") pod \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " May 29 00:55:37.061878 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.061789 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpvm4\" (UniqueName: \"kubernetes.io/projected/a02d348b-9e24-4ea7-b372-1e9d19f7507b-kube-api-access-jpvm4\") pod \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " May 29 00:55:37.061878 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.061811 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-serving-cert\") pod \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " May 29 00:55:37.061878 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.061831 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-config\") pod \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " May 29 00:55:37.062105 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.061887 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-oauth-config\") pod \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " May 29 00:55:37.062105 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.061941 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-oauth-serving-cert\") pod \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " May 29 00:55:37.062105 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.061975 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-trusted-ca-bundle\") pod \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\" (UID: \"a02d348b-9e24-4ea7-b372-1e9d19f7507b\") " May 29 00:55:37.062252 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.062105 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-service-ca" (OuterVolumeSpecName: "service-ca") pod "a02d348b-9e24-4ea7-b372-1e9d19f7507b" (UID: "a02d348b-9e24-4ea7-b372-1e9d19f7507b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:55:37.062310 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.062260 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-service-ca\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:37.062388 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.062363 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-config" (OuterVolumeSpecName: "console-config") pod "a02d348b-9e24-4ea7-b372-1e9d19f7507b" (UID: "a02d348b-9e24-4ea7-b372-1e9d19f7507b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:55:37.062486 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.062457 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a02d348b-9e24-4ea7-b372-1e9d19f7507b" (UID: "a02d348b-9e24-4ea7-b372-1e9d19f7507b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:55:37.062728 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.062707 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a02d348b-9e24-4ea7-b372-1e9d19f7507b" (UID: "a02d348b-9e24-4ea7-b372-1e9d19f7507b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 29 00:55:37.064742 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.064720 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a02d348b-9e24-4ea7-b372-1e9d19f7507b" (UID: "a02d348b-9e24-4ea7-b372-1e9d19f7507b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 00:55:37.064927 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.064898 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a02d348b-9e24-4ea7-b372-1e9d19f7507b" (UID: "a02d348b-9e24-4ea7-b372-1e9d19f7507b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 29 00:55:37.064971 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.064949 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a02d348b-9e24-4ea7-b372-1e9d19f7507b-kube-api-access-jpvm4" (OuterVolumeSpecName: "kube-api-access-jpvm4") pod "a02d348b-9e24-4ea7-b372-1e9d19f7507b" (UID: "a02d348b-9e24-4ea7-b372-1e9d19f7507b"). InnerVolumeSpecName "kube-api-access-jpvm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 00:55:37.162949 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.162928 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-config\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:37.163026 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.162952 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-oauth-config\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:37.163026 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.162965 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-oauth-serving-cert\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:37.163026 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.162976 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02d348b-9e24-4ea7-b372-1e9d19f7507b-trusted-ca-bundle\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:37.163026 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.162985 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpvm4\" (UniqueName: \"kubernetes.io/projected/a02d348b-9e24-4ea7-b372-1e9d19f7507b-kube-api-access-jpvm4\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:37.163026 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.162993 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a02d348b-9e24-4ea7-b372-1e9d19f7507b-console-serving-cert\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:55:37.812053 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.812030 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66f9f94d8f-kbm94_a02d348b-9e24-4ea7-b372-1e9d19f7507b/console/0.log" May 29 00:55:37.812471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.812075 2551 generic.go:358] "Generic (PLEG): container finished" podID="a02d348b-9e24-4ea7-b372-1e9d19f7507b" containerID="ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99" exitCode=2 May 29 00:55:37.812471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.812148 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f9f94d8f-kbm94" May 29 00:55:37.812471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.812162 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f9f94d8f-kbm94" event={"ID":"a02d348b-9e24-4ea7-b372-1e9d19f7507b","Type":"ContainerDied","Data":"ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99"} May 29 00:55:37.812471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.812200 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f9f94d8f-kbm94" event={"ID":"a02d348b-9e24-4ea7-b372-1e9d19f7507b","Type":"ContainerDied","Data":"d6154c99880f90644116c20b64f695e11a16272cd3c96f87abd0c7464e6e49d0"} May 29 00:55:37.812471 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.812225 2551 scope.go:117] "RemoveContainer" containerID="ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99" May 29 00:55:37.821295 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.821276 2551 scope.go:117] "RemoveContainer" containerID="ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99" May 29 00:55:37.821545 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:55:37.821522 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99\": container with ID starting with ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99 not found: ID does not exist" containerID="ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99" May 29 00:55:37.821611 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.821557 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99"} err="failed to get container status \"ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99\": rpc error: code = NotFound desc = could not find container \"ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99\": container with ID starting with ba5aa3093b1fb05e5b8bd25cbeb2303a7b437d6366e4507d4a378b6b55c8ff99 not found: ID does not exist" May 29 00:55:37.837130 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.837108 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66f9f94d8f-kbm94"] May 29 00:55:37.841072 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:37.841050 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66f9f94d8f-kbm94"] May 29 00:55:38.141001 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:38.140947 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a02d348b-9e24-4ea7-b372-1e9d19f7507b" path="/var/lib/kubelet/pods/a02d348b-9e24-4ea7-b372-1e9d19f7507b/volumes" May 29 00:55:38.782764 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:55:38.782735 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 00:56:02.961217 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.961185 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9q4bj"] May 29 00:56:02.961633 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.961525 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a02d348b-9e24-4ea7-b372-1e9d19f7507b" containerName="console" May 29 00:56:02.961633 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.961537 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a02d348b-9e24-4ea7-b372-1e9d19f7507b" containerName="console" May 29 00:56:02.961633 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.961568 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae791456-bbd0-4b2e-ad76-4c3fdde44a59" containerName="manager" May 29 00:56:02.961633 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.961574 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae791456-bbd0-4b2e-ad76-4c3fdde44a59" containerName="manager" May 29 00:56:02.961633 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.961624 2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae791456-bbd0-4b2e-ad76-4c3fdde44a59" containerName="manager" May 29 00:56:02.961633 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.961633 2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="a02d348b-9e24-4ea7-b372-1e9d19f7507b" containerName="console" May 29 00:56:02.966262 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.966239 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:02.969042 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.969018 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" May 29 00:56:02.969959 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.969928 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-r8hgh\"" May 29 00:56:02.970961 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:02.970940 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9q4bj"] May 29 00:56:03.033897 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.033866 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbpl5\" (UniqueName: \"kubernetes.io/projected/b3f0fc41-0785-4898-8852-6a3288befe97-kube-api-access-cbpl5\") pod \"postgres-68c4fbbd6f-9q4bj\" (UID: \"b3f0fc41-0785-4898-8852-6a3288befe97\") " pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:03.034027 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.033902 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b3f0fc41-0785-4898-8852-6a3288befe97-data\") pod \"postgres-68c4fbbd6f-9q4bj\" (UID: \"b3f0fc41-0785-4898-8852-6a3288befe97\") " pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:03.134911 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.134878 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cbpl5\" (UniqueName: \"kubernetes.io/projected/b3f0fc41-0785-4898-8852-6a3288befe97-kube-api-access-cbpl5\") pod \"postgres-68c4fbbd6f-9q4bj\" (UID: \"b3f0fc41-0785-4898-8852-6a3288befe97\") " pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:03.134911 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.134913 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b3f0fc41-0785-4898-8852-6a3288befe97-data\") pod \"postgres-68c4fbbd6f-9q4bj\" (UID: \"b3f0fc41-0785-4898-8852-6a3288befe97\") " pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:03.135238 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.135223 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b3f0fc41-0785-4898-8852-6a3288befe97-data\") pod \"postgres-68c4fbbd6f-9q4bj\" (UID: \"b3f0fc41-0785-4898-8852-6a3288befe97\") " pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:03.142571 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.142543 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbpl5\" (UniqueName: \"kubernetes.io/projected/b3f0fc41-0785-4898-8852-6a3288befe97-kube-api-access-cbpl5\") pod \"postgres-68c4fbbd6f-9q4bj\" (UID: \"b3f0fc41-0785-4898-8852-6a3288befe97\") " pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:03.279666 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.279638 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:03.404084 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.404059 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9q4bj"] May 29 00:56:03.406046 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:56:03.406024 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f0fc41_0785_4898_8852_6a3288befe97.slice/crio-405054fcf1f03ba025932cf3518ef6db16a11d387f759f5f7ef2c13f9ab5d199 WatchSource:0}: Error finding container 405054fcf1f03ba025932cf3518ef6db16a11d387f759f5f7ef2c13f9ab5d199: Status 404 returned error can't find the container with id 405054fcf1f03ba025932cf3518ef6db16a11d387f759f5f7ef2c13f9ab5d199 May 29 00:56:03.913099 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:03.913063 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-9q4bj" event={"ID":"b3f0fc41-0785-4898-8852-6a3288befe97","Type":"ContainerStarted","Data":"405054fcf1f03ba025932cf3518ef6db16a11d387f759f5f7ef2c13f9ab5d199"} May 29 00:56:08.939340 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:08.939303 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-9q4bj" event={"ID":"b3f0fc41-0785-4898-8852-6a3288befe97","Type":"ContainerStarted","Data":"308226e982771c4dcfde343d9479f2400f1c0e7be9e765c3dfbcc0ff9aa4c460"} May 29 00:56:08.939700 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:08.939360 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:56:08.956164 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:08.956117 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-9q4bj" podStartSLOduration=1.990683269 podStartE2EDuration="6.956100666s" podCreationTimestamp="2026-05-29 00:56:02 +0000 UTC" firstStartedPulling="2026-05-29 00:56:03.4076828 +0000 UTC m=+521.867886228" lastFinishedPulling="2026-05-29 00:56:08.373100181 +0000 UTC m=+526.833303625" observedRunningTime="2026-05-29 00:56:08.954611146 +0000 UTC m=+527.414814597" watchObservedRunningTime="2026-05-29 00:56:08.956100666 +0000 UTC m=+527.416304117" May 29 00:56:14.970693 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:56:14.970663 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-9q4bj" May 29 00:57:13.164651 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.164609 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b6cb86dc4-ttq8t"] May 29 00:57:13.168717 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.168696 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:13.171214 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.171194 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 29 00:57:13.171351 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.171334 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6bbf4\"" May 29 00:57:13.176632 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.176175 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b6cb86dc4-ttq8t"] May 29 00:57:13.275829 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.275803 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sbsj\" (UniqueName: \"kubernetes.io/projected/0f544f43-f1bb-411e-8daa-60d35636d3d4-kube-api-access-9sbsj\") pod \"maas-controller-5b6cb86dc4-ttq8t\" (UID: \"0f544f43-f1bb-411e-8daa-60d35636d3d4\") " pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:13.310580 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.310552 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-9ztl8"] May 29 00:57:13.313879 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.313859 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 00:57:13.320962 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.320941 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-9ztl8"] May 29 00:57:13.376629 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.376605 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9sbsj\" (UniqueName: \"kubernetes.io/projected/0f544f43-f1bb-411e-8daa-60d35636d3d4-kube-api-access-9sbsj\") pod \"maas-controller-5b6cb86dc4-ttq8t\" (UID: \"0f544f43-f1bb-411e-8daa-60d35636d3d4\") " pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:13.385832 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.385811 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sbsj\" (UniqueName: \"kubernetes.io/projected/0f544f43-f1bb-411e-8daa-60d35636d3d4-kube-api-access-9sbsj\") pod \"maas-controller-5b6cb86dc4-ttq8t\" (UID: \"0f544f43-f1bb-411e-8daa-60d35636d3d4\") " pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:13.477103 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.477081 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxp7z\" (UniqueName: \"kubernetes.io/projected/1bff1537-1058-47dc-a292-d894a499367d-kube-api-access-pxp7z\") pod \"maas-controller-6b9fdf5dcf-9ztl8\" (UID: \"1bff1537-1058-47dc-a292-d894a499367d\") " pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 00:57:13.480138 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.480120 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:13.578543 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.578511 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxp7z\" (UniqueName: \"kubernetes.io/projected/1bff1537-1058-47dc-a292-d894a499367d-kube-api-access-pxp7z\") pod \"maas-controller-6b9fdf5dcf-9ztl8\" (UID: \"1bff1537-1058-47dc-a292-d894a499367d\") " pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 00:57:13.586869 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.586843 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxp7z\" (UniqueName: \"kubernetes.io/projected/1bff1537-1058-47dc-a292-d894a499367d-kube-api-access-pxp7z\") pod \"maas-controller-6b9fdf5dcf-9ztl8\" (UID: \"1bff1537-1058-47dc-a292-d894a499367d\") " pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 00:57:13.602482 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.602457 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b6cb86dc4-ttq8t"] May 29 00:57:13.605710 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:57:13.605684 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f544f43_f1bb_411e_8daa_60d35636d3d4.slice/crio-68bf748b0633c5e75ade2882fe3670212e721b21a5fa57174a638a3e7bda3fd9 WatchSource:0}: Error finding container 68bf748b0633c5e75ade2882fe3670212e721b21a5fa57174a638a3e7bda3fd9: Status 404 returned error can't find the container with id 68bf748b0633c5e75ade2882fe3670212e721b21a5fa57174a638a3e7bda3fd9 May 29 00:57:13.624539 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.624511 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 00:57:13.744697 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:13.744672 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-9ztl8"] May 29 00:57:13.746493 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:57:13.746466 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bff1537_1058_47dc_a292_d894a499367d.slice/crio-c220ca40b15ac392e2a4301d82e9ae08b3cc88d15571a0126b0580a95494ab32 WatchSource:0}: Error finding container c220ca40b15ac392e2a4301d82e9ae08b3cc88d15571a0126b0580a95494ab32: Status 404 returned error can't find the container with id c220ca40b15ac392e2a4301d82e9ae08b3cc88d15571a0126b0580a95494ab32 May 29 00:57:14.163860 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:14.163787 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" event={"ID":"0f544f43-f1bb-411e-8daa-60d35636d3d4","Type":"ContainerStarted","Data":"68bf748b0633c5e75ade2882fe3670212e721b21a5fa57174a638a3e7bda3fd9"} May 29 00:57:14.165115 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:14.165082 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" event={"ID":"1bff1537-1058-47dc-a292-d894a499367d","Type":"ContainerStarted","Data":"c220ca40b15ac392e2a4301d82e9ae08b3cc88d15571a0126b0580a95494ab32"} May 29 00:57:17.179812 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:17.179748 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" event={"ID":"0f544f43-f1bb-411e-8daa-60d35636d3d4","Type":"ContainerStarted","Data":"7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26"} May 29 00:57:17.180239 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:17.179828 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:17.181045 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:17.181023 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" event={"ID":"1bff1537-1058-47dc-a292-d894a499367d","Type":"ContainerStarted","Data":"bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43"} May 29 00:57:17.181160 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:17.181145 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 00:57:17.195647 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:17.195599 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" podStartSLOduration=1.50584324 podStartE2EDuration="4.195585408s" podCreationTimestamp="2026-05-29 00:57:13 +0000 UTC" firstStartedPulling="2026-05-29 00:57:13.606930187 +0000 UTC m=+592.067133615" lastFinishedPulling="2026-05-29 00:57:16.29667235 +0000 UTC m=+594.756875783" observedRunningTime="2026-05-29 00:57:17.194814037 +0000 UTC m=+595.655017491" watchObservedRunningTime="2026-05-29 00:57:17.195585408 +0000 UTC m=+595.655788859" May 29 00:57:17.210280 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:17.210245 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" podStartSLOduration=1.650227604 podStartE2EDuration="4.210234439s" podCreationTimestamp="2026-05-29 00:57:13 +0000 UTC" firstStartedPulling="2026-05-29 00:57:13.747848046 +0000 UTC m=+592.208051474" lastFinishedPulling="2026-05-29 00:57:16.307854881 +0000 UTC m=+594.768058309" observedRunningTime="2026-05-29 00:57:17.209274462 +0000 UTC m=+595.669477912" watchObservedRunningTime="2026-05-29 00:57:17.210234439 +0000 UTC m=+595.670437889" May 29 00:57:22.081114 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:22.081080 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 00:57:22.081519 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:22.081312 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 00:57:28.189814 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:28.189788 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:28.192304 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:28.189840 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 00:57:28.247770 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:28.244611 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b6cb86dc4-ttq8t"] May 29 00:57:28.247770 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:28.244936 2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" podUID="0f544f43-f1bb-411e-8daa-60d35636d3d4" containerName="manager" containerID="cri-o://7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26" gracePeriod=30 May 29 00:57:28.491242 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:28.491215 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:28.596092 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:28.596068 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sbsj\" (UniqueName: \"kubernetes.io/projected/0f544f43-f1bb-411e-8daa-60d35636d3d4-kube-api-access-9sbsj\") pod \"0f544f43-f1bb-411e-8daa-60d35636d3d4\" (UID: \"0f544f43-f1bb-411e-8daa-60d35636d3d4\") " May 29 00:57:28.598101 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:28.598069 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f544f43-f1bb-411e-8daa-60d35636d3d4-kube-api-access-9sbsj" (OuterVolumeSpecName: "kube-api-access-9sbsj") pod "0f544f43-f1bb-411e-8daa-60d35636d3d4" (UID: "0f544f43-f1bb-411e-8daa-60d35636d3d4"). InnerVolumeSpecName "kube-api-access-9sbsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 00:57:28.696591 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:28.696566 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sbsj\" (UniqueName: \"kubernetes.io/projected/0f544f43-f1bb-411e-8daa-60d35636d3d4-kube-api-access-9sbsj\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 00:57:29.220450 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.220417 2551 generic.go:358] "Generic (PLEG): container finished" podID="0f544f43-f1bb-411e-8daa-60d35636d3d4" containerID="7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26" exitCode=0 May 29 00:57:29.220984 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.220478 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" May 29 00:57:29.220984 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.220493 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" event={"ID":"0f544f43-f1bb-411e-8daa-60d35636d3d4","Type":"ContainerDied","Data":"7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26"} May 29 00:57:29.220984 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.220521 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6cb86dc4-ttq8t" event={"ID":"0f544f43-f1bb-411e-8daa-60d35636d3d4","Type":"ContainerDied","Data":"68bf748b0633c5e75ade2882fe3670212e721b21a5fa57174a638a3e7bda3fd9"} May 29 00:57:29.220984 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.220535 2551 scope.go:117] "RemoveContainer" containerID="7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26" May 29 00:57:29.229379 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.229363 2551 scope.go:117] "RemoveContainer" containerID="7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26" May 29 00:57:29.229610 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:57:29.229594 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26\": container with ID starting with 7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26 not found: ID does not exist" containerID="7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26" May 29 00:57:29.229662 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.229618 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26"} err="failed to get container status \"7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26\": rpc error: code = NotFound desc = could not find container \"7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26\": container with ID starting with 7bc75ade6433c661fada879325d2c0ec0460af25bfef20472158088989881c26 not found: ID does not exist" May 29 00:57:29.241073 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.241046 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b6cb86dc4-ttq8t"] May 29 00:57:29.243297 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:29.243278 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b6cb86dc4-ttq8t"] May 29 00:57:30.141326 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:30.141290 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f544f43-f1bb-411e-8daa-60d35636d3d4" path="/var/lib/kubelet/pods/0f544f43-f1bb-411e-8daa-60d35636d3d4/volumes" May 29 00:57:47.454639 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.454569 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-586b46cd66-q6m9v"] May 29 00:57:47.455013 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.454996 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f544f43-f1bb-411e-8daa-60d35636d3d4" containerName="manager" May 29 00:57:47.455055 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.455015 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f544f43-f1bb-411e-8daa-60d35636d3d4" containerName="manager" May 29 00:57:47.455092 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.455078 2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="0f544f43-f1bb-411e-8daa-60d35636d3d4" containerName="manager" May 29 00:57:47.458223 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.458207 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:47.460846 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.460824 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" May 29 00:57:47.465707 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.465685 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-586b46cd66-q6m9v"] May 29 00:57:47.539795 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.539761 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nctx6\" (UniqueName: \"kubernetes.io/projected/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-kube-api-access-nctx6\") pod \"maas-api-586b46cd66-q6m9v\" (UID: \"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8\") " pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:47.539940 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.539816 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-maas-api-tls\") pod \"maas-api-586b46cd66-q6m9v\" (UID: \"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8\") " pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:47.640307 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.640276 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-maas-api-tls\") pod \"maas-api-586b46cd66-q6m9v\" (UID: \"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8\") " pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:47.640441 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.640364 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nctx6\" (UniqueName: \"kubernetes.io/projected/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-kube-api-access-nctx6\") pod \"maas-api-586b46cd66-q6m9v\" (UID: \"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8\") " pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:47.640441 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:57:47.640425 2551 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found May 29 00:57:47.640529 ip-10-0-141-236 kubenswrapper[2551]: E0529 00:57:47.640488 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-maas-api-tls podName:343cd0c6-a66e-4621-9433-b0cc0d0c6ae8 nodeName:}" failed. No retries permitted until 2026-05-29 00:57:48.14046948 +0000 UTC m=+626.600672913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-maas-api-tls") pod "maas-api-586b46cd66-q6m9v" (UID: "343cd0c6-a66e-4621-9433-b0cc0d0c6ae8") : secret "maas-api-serving-cert" not found May 29 00:57:47.649406 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:47.649384 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nctx6\" (UniqueName: \"kubernetes.io/projected/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-kube-api-access-nctx6\") pod \"maas-api-586b46cd66-q6m9v\" (UID: \"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8\") " pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:48.145394 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:48.145363 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-maas-api-tls\") pod \"maas-api-586b46cd66-q6m9v\" (UID: \"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8\") " pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:48.148045 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:48.148012 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343cd0c6-a66e-4621-9433-b0cc0d0c6ae8-maas-api-tls\") pod \"maas-api-586b46cd66-q6m9v\" (UID: \"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8\") " pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:48.370136 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:48.370102 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:48.502124 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:48.502093 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-586b46cd66-q6m9v"] May 29 00:57:48.504304 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:57:48.504274 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343cd0c6_a66e_4621_9433_b0cc0d0c6ae8.slice/crio-1ec58a6b03e09e09fec49646fc006102b3287ef38d53c853e61d211cc31f37d5 WatchSource:0}: Error finding container 1ec58a6b03e09e09fec49646fc006102b3287ef38d53c853e61d211cc31f37d5: Status 404 returned error can't find the container with id 1ec58a6b03e09e09fec49646fc006102b3287ef38d53c853e61d211cc31f37d5 May 29 00:57:48.505534 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:48.505518 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 00:57:49.287714 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:49.287680 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-586b46cd66-q6m9v" event={"ID":"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8","Type":"ContainerStarted","Data":"1ec58a6b03e09e09fec49646fc006102b3287ef38d53c853e61d211cc31f37d5"} May 29 00:57:51.297302 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:51.297255 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-586b46cd66-q6m9v" event={"ID":"343cd0c6-a66e-4621-9433-b0cc0d0c6ae8","Type":"ContainerStarted","Data":"34e180ea452730476a6f0d9a9c25f8ecd8a95c7c60d461479b7bbe60f5ea92f1"} May 29 00:57:51.297732 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:51.297432 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:57:51.314231 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:51.314187 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-586b46cd66-q6m9v" podStartSLOduration=2.496213413 podStartE2EDuration="4.314166711s" podCreationTimestamp="2026-05-29 00:57:47 +0000 UTC" firstStartedPulling="2026-05-29 00:57:48.505649727 +0000 UTC m=+626.965853159" lastFinishedPulling="2026-05-29 00:57:50.323603022 +0000 UTC m=+628.783806457" observedRunningTime="2026-05-29 00:57:51.312159813 +0000 UTC m=+629.772363262" watchObservedRunningTime="2026-05-29 00:57:51.314166711 +0000 UTC m=+629.774370162" May 29 00:57:57.305969 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:57:57.305941 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-586b46cd66-q6m9v" May 29 00:58:50.687655 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.687618 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d"] May 29 00:58:50.691270 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.691248 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.694088 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.694067 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 29 00:58:50.695277 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.695260 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 29 00:58:50.695399 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.695381 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5q8n9\"" May 29 00:58:50.695473 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.695382 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" May 29 00:58:50.702149 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.702131 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d"] May 29 00:58:50.730599 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.730439 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78db6ad4-edef-4d40-ab94-1295d43ee7b2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.730599 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.730496 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.730599 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.730530 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.730599 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.730562 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.730599 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.730587 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94klx\" (UniqueName: \"kubernetes.io/projected/78db6ad4-edef-4d40-ab94-1295d43ee7b2-kube-api-access-94klx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.730938 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.730635 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.831440 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.831417 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.832007 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.831989 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.832390 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.832376 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-94klx\" (UniqueName: \"kubernetes.io/projected/78db6ad4-edef-4d40-ab94-1295d43ee7b2-kube-api-access-94klx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.832816 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.832798 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.833008 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.832993 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78db6ad4-edef-4d40-ab94-1295d43ee7b2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.833448 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.833432 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.833582 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.831928 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.833582 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.832334 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.833829 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.833810 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.836220 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.836200 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78db6ad4-edef-4d40-ab94-1295d43ee7b2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.836930 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.836905 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78db6ad4-edef-4d40-ab94-1295d43ee7b2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:50.839951 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:50.839928 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94klx\" (UniqueName: \"kubernetes.io/projected/78db6ad4-edef-4d40-ab94-1295d43ee7b2-kube-api-access-94klx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d\" (UID: \"78db6ad4-edef-4d40-ab94-1295d43ee7b2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:51.002493 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:51.002469 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:58:51.138845 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:51.138824 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d"] May 29 00:58:51.141856 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:58:51.141825 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78db6ad4_edef_4d40_ab94_1295d43ee7b2.slice/crio-ee84ee3757a9fe5b3ecd1170c2568d0edeafc58f73d8237b8026de6061567dfb WatchSource:0}: Error finding container ee84ee3757a9fe5b3ecd1170c2568d0edeafc58f73d8237b8026de6061567dfb: Status 404 returned error can't find the container with id ee84ee3757a9fe5b3ecd1170c2568d0edeafc58f73d8237b8026de6061567dfb May 29 00:58:51.509300 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:51.509265 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" event={"ID":"78db6ad4-edef-4d40-ab94-1295d43ee7b2","Type":"ContainerStarted","Data":"ee84ee3757a9fe5b3ecd1170c2568d0edeafc58f73d8237b8026de6061567dfb"} May 29 00:58:58.548588 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:58.548499 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" event={"ID":"78db6ad4-edef-4d40-ab94-1295d43ee7b2","Type":"ContainerStarted","Data":"5412a2d9621ab87181bb78c35d5562d973980da2a3b1e48fdd0ee64fadb22db9"} May 29 00:58:59.771363 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.771311 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6"] May 29 00:58:59.775397 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.775372 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.778221 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.778115 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" May 29 00:58:59.784125 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.784104 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6"] May 29 00:58:59.824446 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.824413 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.824592 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.824491 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.824592 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.824575 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zppsz\" (UniqueName: \"kubernetes.io/projected/03ca099b-6665-4598-8fed-cb9136ca343e-kube-api-access-zppsz\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.824665 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.824624 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03ca099b-6665-4598-8fed-cb9136ca343e-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.824700 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.824666 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.824737 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.824711 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.925315 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.925275 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.925464 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.925322 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.925505 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.925458 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zppsz\" (UniqueName: \"kubernetes.io/projected/03ca099b-6665-4598-8fed-cb9136ca343e-kube-api-access-zppsz\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.925542 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.925499 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03ca099b-6665-4598-8fed-cb9136ca343e-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.925576 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.925549 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.925613 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.925583 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.926072 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.926052 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.926129 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.926099 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.926421 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.926397 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.927931 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.927905 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03ca099b-6665-4598-8fed-cb9136ca343e-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.928302 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.928282 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03ca099b-6665-4598-8fed-cb9136ca343e-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:58:59.940315 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:58:59.940294 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zppsz\" (UniqueName: \"kubernetes.io/projected/03ca099b-6665-4598-8fed-cb9136ca343e-kube-api-access-zppsz\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6\" (UID: \"03ca099b-6665-4598-8fed-cb9136ca343e\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:59:00.086592 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:00.086513 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:59:00.229270 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:00.229246 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6"] May 29 00:59:00.557632 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:00.557588 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" event={"ID":"03ca099b-6665-4598-8fed-cb9136ca343e","Type":"ContainerStarted","Data":"d3a7b156fd0b8dd0ee5cfa2780286b4efe29eed289c31ede352a165417b975ae"} May 29 00:59:00.557632 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:00.557630 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" event={"ID":"03ca099b-6665-4598-8fed-cb9136ca343e","Type":"ContainerStarted","Data":"2adc50febe2fe21f35a383f692ff4fab8c7c3c1b3d13eee7fc28873925edb7bb"} May 29 00:59:03.570249 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:03.570149 2551 generic.go:358] "Generic (PLEG): container finished" podID="78db6ad4-edef-4d40-ab94-1295d43ee7b2" containerID="5412a2d9621ab87181bb78c35d5562d973980da2a3b1e48fdd0ee64fadb22db9" exitCode=0 May 29 00:59:03.570674 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:03.570273 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" event={"ID":"78db6ad4-edef-4d40-ab94-1295d43ee7b2","Type":"ContainerDied","Data":"5412a2d9621ab87181bb78c35d5562d973980da2a3b1e48fdd0ee64fadb22db9"} May 29 00:59:05.579828 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:05.579793 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" event={"ID":"78db6ad4-edef-4d40-ab94-1295d43ee7b2","Type":"ContainerStarted","Data":"b30dcfd7794e824e7584c6e840475c4af492fd3dbd6c9ef0e44bacb8010ab4f3"} May 29 00:59:05.580267 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:05.580038 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:59:05.598535 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:05.598486 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" podStartSLOduration=1.9148591590000001 podStartE2EDuration="15.598470796s" podCreationTimestamp="2026-05-29 00:58:50 +0000 UTC" firstStartedPulling="2026-05-29 00:58:51.143916064 +0000 UTC m=+689.604119493" lastFinishedPulling="2026-05-29 00:59:04.827527689 +0000 UTC m=+703.287731130" observedRunningTime="2026-05-29 00:59:05.596382406 +0000 UTC m=+704.056585847" watchObservedRunningTime="2026-05-29 00:59:05.598470796 +0000 UTC m=+704.058674246" May 29 00:59:06.584885 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:06.584848 2551 generic.go:358] "Generic (PLEG): container finished" podID="03ca099b-6665-4598-8fed-cb9136ca343e" containerID="d3a7b156fd0b8dd0ee5cfa2780286b4efe29eed289c31ede352a165417b975ae" exitCode=0 May 29 00:59:06.585298 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:06.584929 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" event={"ID":"03ca099b-6665-4598-8fed-cb9136ca343e","Type":"ContainerDied","Data":"d3a7b156fd0b8dd0ee5cfa2780286b4efe29eed289c31ede352a165417b975ae"} May 29 00:59:07.591380 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:07.591347 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" event={"ID":"03ca099b-6665-4598-8fed-cb9136ca343e","Type":"ContainerStarted","Data":"790916d5cac5dc5f15adc3e9898e9902ef19a316a107912f6e69c6e0a25a8e91"} May 29 00:59:07.591731 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:07.591569 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:59:07.610013 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:07.609967 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" podStartSLOduration=8.430852358 podStartE2EDuration="8.609953711s" podCreationTimestamp="2026-05-29 00:58:59 +0000 UTC" firstStartedPulling="2026-05-29 00:59:06.585728268 +0000 UTC m=+705.045931697" lastFinishedPulling="2026-05-29 00:59:06.764829621 +0000 UTC m=+705.225033050" observedRunningTime="2026-05-29 00:59:07.608837352 +0000 UTC m=+706.069040802" watchObservedRunningTime="2026-05-29 00:59:07.609953711 +0000 UTC m=+706.070157229" May 29 00:59:08.078941 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.078906 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2"] May 29 00:59:08.084227 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.084202 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.086870 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.086844 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" May 29 00:59:08.095047 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.095027 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2"] May 29 00:59:08.202959 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.202928 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.203118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.202978 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.203118 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.203085 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.203227 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.203120 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.203227 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.203139 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.203314 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.203220 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvkmb\" (UniqueName: \"kubernetes.io/projected/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-kube-api-access-gvkmb\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304184 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304153 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304342 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304193 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304342 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304221 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304342 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304259 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gvkmb\" (UniqueName: \"kubernetes.io/projected/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-kube-api-access-gvkmb\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304342 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304291 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304342 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304333 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304604 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304577 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304661 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304613 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.304735 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.304711 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.306331 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.306307 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.306618 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.306602 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.311274 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.311252 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvkmb\" (UniqueName: \"kubernetes.io/projected/0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc-kube-api-access-gvkmb\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-kkgn2\" (UID: \"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.397212 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.397139 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:08.530483 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.530450 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2"] May 29 00:59:08.533597 ip-10-0-141-236 kubenswrapper[2551]: W0529 00:59:08.533568 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee30f30_9d2e_4e84_a2e1_9f75859cf9bc.slice/crio-9a703db7af3e1fea91fc4d8df8c3aef94f4563ed46b9652e98ff23966497de5a WatchSource:0}: Error finding container 9a703db7af3e1fea91fc4d8df8c3aef94f4563ed46b9652e98ff23966497de5a: Status 404 returned error can't find the container with id 9a703db7af3e1fea91fc4d8df8c3aef94f4563ed46b9652e98ff23966497de5a May 29 00:59:08.598547 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.598519 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" event={"ID":"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc","Type":"ContainerStarted","Data":"34e4df8e9f3c77c6ac66353f9dcbed983f1dd35eaf5f6459bd9ea6a43c6a8720"} May 29 00:59:08.598905 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:08.598557 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" event={"ID":"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc","Type":"ContainerStarted","Data":"9a703db7af3e1fea91fc4d8df8c3aef94f4563ed46b9652e98ff23966497de5a"} May 29 00:59:14.622770 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:14.622713 2551 generic.go:358] "Generic (PLEG): container finished" podID="0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc" containerID="34e4df8e9f3c77c6ac66353f9dcbed983f1dd35eaf5f6459bd9ea6a43c6a8720" exitCode=0 May 29 00:59:14.623133 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:14.622790 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" event={"ID":"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc","Type":"ContainerDied","Data":"34e4df8e9f3c77c6ac66353f9dcbed983f1dd35eaf5f6459bd9ea6a43c6a8720"} May 29 00:59:15.628734 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:15.628703 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" event={"ID":"0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc","Type":"ContainerStarted","Data":"4e0e3bcde6ff84e85d08d6c5a1e1dfa78413e0982fff733934bf9e5550e2921b"} May 29 00:59:15.629074 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:15.628937 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 00:59:15.647721 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:15.647670 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" podStartSLOduration=7.231367002 podStartE2EDuration="7.647657485s" podCreationTimestamp="2026-05-29 00:59:08 +0000 UTC" firstStartedPulling="2026-05-29 00:59:14.623402751 +0000 UTC m=+713.083606179" lastFinishedPulling="2026-05-29 00:59:15.039693217 +0000 UTC m=+713.499896662" observedRunningTime="2026-05-29 00:59:15.645883998 +0000 UTC m=+714.106087460" watchObservedRunningTime="2026-05-29 00:59:15.647657485 +0000 UTC m=+714.107860946" May 29 00:59:16.597485 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:16.597456 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d" May 29 00:59:18.610898 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:18.610870 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6" May 29 00:59:26.648181 ip-10-0-141-236 kubenswrapper[2551]: I0529 00:59:26.648153 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-kkgn2" May 29 01:01:26.733599 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:26.733549 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-9ztl8"] May 29 01:01:26.734064 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:26.733842 2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" podUID="1bff1537-1058-47dc-a292-d894a499367d" containerName="manager" containerID="cri-o://bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43" gracePeriod=30 May 29 01:01:26.980735 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:26.980713 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 01:01:27.081810 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.081725 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxp7z\" (UniqueName: \"kubernetes.io/projected/1bff1537-1058-47dc-a292-d894a499367d-kube-api-access-pxp7z\") pod \"1bff1537-1058-47dc-a292-d894a499367d\" (UID: \"1bff1537-1058-47dc-a292-d894a499367d\") " May 29 01:01:27.083682 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.083653 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bff1537-1058-47dc-a292-d894a499367d-kube-api-access-pxp7z" (OuterVolumeSpecName: "kube-api-access-pxp7z") pod "1bff1537-1058-47dc-a292-d894a499367d" (UID: "1bff1537-1058-47dc-a292-d894a499367d"). InnerVolumeSpecName "kube-api-access-pxp7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 01:01:27.104993 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.104965 2551 generic.go:358] "Generic (PLEG): container finished" podID="1bff1537-1058-47dc-a292-d894a499367d" containerID="bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43" exitCode=0 May 29 01:01:27.105093 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.105041 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" May 29 01:01:27.105093 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.105047 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" event={"ID":"1bff1537-1058-47dc-a292-d894a499367d","Type":"ContainerDied","Data":"bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43"} May 29 01:01:27.105093 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.105079 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9fdf5dcf-9ztl8" event={"ID":"1bff1537-1058-47dc-a292-d894a499367d","Type":"ContainerDied","Data":"c220ca40b15ac392e2a4301d82e9ae08b3cc88d15571a0126b0580a95494ab32"} May 29 01:01:27.105251 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.105096 2551 scope.go:117] "RemoveContainer" containerID="bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43" May 29 01:01:27.114101 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.114067 2551 scope.go:117] "RemoveContainer" containerID="bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43" May 29 01:01:27.114343 ip-10-0-141-236 kubenswrapper[2551]: E0529 01:01:27.114323 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43\": container with ID starting with bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43 not found: ID does not exist" containerID="bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43" May 29 01:01:27.114412 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.114353 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43"} err="failed to get container status \"bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43\": rpc error: code = NotFound desc = could not find container \"bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43\": container with ID starting with bac8cf2bcf9dc0fe4bd84519d70e6f6aa461e62acbcbd7396ca00fc920de8a43 not found: ID does not exist" May 29 01:01:27.127807 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.127786 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-9ztl8"] May 29 01:01:27.129405 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.129385 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-9ztl8"] May 29 01:01:27.182610 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.182582 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxp7z\" (UniqueName: \"kubernetes.io/projected/1bff1537-1058-47dc-a292-d894a499367d-kube-api-access-pxp7z\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 01:01:27.849746 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.849709 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-xjdj5"] May 29 01:01:27.850274 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.850254 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bff1537-1058-47dc-a292-d894a499367d" containerName="manager" May 29 01:01:27.850355 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.850275 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bff1537-1058-47dc-a292-d894a499367d" containerName="manager" May 29 01:01:27.850408 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.850398 2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="1bff1537-1058-47dc-a292-d894a499367d" containerName="manager" May 29 01:01:27.854182 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.854155 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" May 29 01:01:27.857028 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.856975 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 29 01:01:27.857143 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.856973 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6bbf4\"" May 29 01:01:27.860708 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.860685 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-xjdj5"] May 29 01:01:27.888892 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.888862 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flncn\" (UniqueName: \"kubernetes.io/projected/7192fc54-a6c8-44f0-95c9-23203c5e435d-kube-api-access-flncn\") pod \"maas-controller-6b9fdf5dcf-xjdj5\" (UID: \"7192fc54-a6c8-44f0-95c9-23203c5e435d\") " pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" May 29 01:01:27.989530 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.989502 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-flncn\" (UniqueName: \"kubernetes.io/projected/7192fc54-a6c8-44f0-95c9-23203c5e435d-kube-api-access-flncn\") pod \"maas-controller-6b9fdf5dcf-xjdj5\" (UID: \"7192fc54-a6c8-44f0-95c9-23203c5e435d\") " pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" May 29 01:01:27.997140 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:27.997114 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flncn\" (UniqueName: \"kubernetes.io/projected/7192fc54-a6c8-44f0-95c9-23203c5e435d-kube-api-access-flncn\") pod \"maas-controller-6b9fdf5dcf-xjdj5\" (UID: \"7192fc54-a6c8-44f0-95c9-23203c5e435d\") " pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" May 29 01:01:28.141691 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:28.141601 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bff1537-1058-47dc-a292-d894a499367d" path="/var/lib/kubelet/pods/1bff1537-1058-47dc-a292-d894a499367d/volumes" May 29 01:01:28.165802 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:28.165773 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" May 29 01:01:28.294335 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:28.294308 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9fdf5dcf-xjdj5"] May 29 01:01:28.295298 ip-10-0-141-236 kubenswrapper[2551]: W0529 01:01:28.295270 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7192fc54_a6c8_44f0_95c9_23203c5e435d.slice/crio-4e167257fe216e7bb0a7b7237594d193c722f7f39f923ba7dd333c51a175d68a WatchSource:0}: Error finding container 4e167257fe216e7bb0a7b7237594d193c722f7f39f923ba7dd333c51a175d68a: Status 404 returned error can't find the container with id 4e167257fe216e7bb0a7b7237594d193c722f7f39f923ba7dd333c51a175d68a May 29 01:01:29.119617 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:29.114092 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" event={"ID":"7192fc54-a6c8-44f0-95c9-23203c5e435d","Type":"ContainerStarted","Data":"6906f9ee1a6e83ceebc9b6bbe1f813eb94abf644c36be81703f1b9486d2eeced"} May 29 01:01:29.119617 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:29.114146 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" event={"ID":"7192fc54-a6c8-44f0-95c9-23203c5e435d","Type":"ContainerStarted","Data":"4e167257fe216e7bb0a7b7237594d193c722f7f39f923ba7dd333c51a175d68a"} May 29 01:01:29.119617 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:29.115207 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" May 29 01:01:29.133268 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:29.133219 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" podStartSLOduration=1.648582855 podStartE2EDuration="2.133204109s" podCreationTimestamp="2026-05-29 01:01:27 +0000 UTC" firstStartedPulling="2026-05-29 01:01:28.296479755 +0000 UTC m=+846.756683183" lastFinishedPulling="2026-05-29 01:01:28.781101006 +0000 UTC m=+847.241304437" observedRunningTime="2026-05-29 01:01:29.131015101 +0000 UTC m=+847.591218550" watchObservedRunningTime="2026-05-29 01:01:29.133204109 +0000 UTC m=+847.593407559" May 29 01:01:41.127260 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:01:41.127229 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b9fdf5dcf-xjdj5" May 29 01:02:22.107344 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:02:22.107259 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:02:22.110781 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:02:22.110742 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:07:22.137523 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:07:22.137491 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:07:22.138621 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:07:22.138601 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:11:48.941090 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:48.941053 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c"] May 29 01:11:48.941686 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:48.941289 2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" podUID="fa9e73e4-3610-49d3-9b61-04d81bf88fb0" containerName="manager" containerID="cri-o://f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47" gracePeriod=10 May 29 01:11:49.691477 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:49.691455 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 01:11:49.810794 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:49.810712 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxt4b\" (UniqueName: \"kubernetes.io/projected/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-kube-api-access-wxt4b\") pod \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\" (UID: \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\") " May 29 01:11:49.810909 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:49.810833 2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-extensions-socket-volume\") pod \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\" (UID: \"fa9e73e4-3610-49d3-9b61-04d81bf88fb0\") " May 29 01:11:49.811172 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:49.811148 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fa9e73e4-3610-49d3-9b61-04d81bf88fb0" (UID: "fa9e73e4-3610-49d3-9b61-04d81bf88fb0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 29 01:11:49.812686 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:49.812667 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-kube-api-access-wxt4b" (OuterVolumeSpecName: "kube-api-access-wxt4b") pod "fa9e73e4-3610-49d3-9b61-04d81bf88fb0" (UID: "fa9e73e4-3610-49d3-9b61-04d81bf88fb0"). InnerVolumeSpecName "kube-api-access-wxt4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 29 01:11:49.912371 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:49.912344 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxt4b\" (UniqueName: \"kubernetes.io/projected/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-kube-api-access-wxt4b\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 01:11:49.912371 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:49.912366 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa9e73e4-3610-49d3-9b61-04d81bf88fb0-extensions-socket-volume\") on node \"ip-10-0-141-236.ec2.internal\" DevicePath \"\"" May 29 01:11:50.353503 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.353464 2551 generic.go:358] "Generic (PLEG): container finished" podID="fa9e73e4-3610-49d3-9b61-04d81bf88fb0" containerID="f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47" exitCode=0 May 29 01:11:50.353894 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.353524 2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" May 29 01:11:50.353894 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.353554 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" event={"ID":"fa9e73e4-3610-49d3-9b61-04d81bf88fb0","Type":"ContainerDied","Data":"f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47"} May 29 01:11:50.353894 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.353589 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c" event={"ID":"fa9e73e4-3610-49d3-9b61-04d81bf88fb0","Type":"ContainerDied","Data":"900aab3ee092a8f4d07ac581659379f15bc5fdb6aeadc868d32fb3356375236e"} May 29 01:11:50.353894 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.353605 2551 scope.go:117] "RemoveContainer" containerID="f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47" May 29 01:11:50.362327 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.362307 2551 scope.go:117] "RemoveContainer" containerID="f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47" May 29 01:11:50.362575 ip-10-0-141-236 kubenswrapper[2551]: E0529 01:11:50.362552 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47\": container with ID starting with f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47 not found: ID does not exist" containerID="f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47" May 29 01:11:50.362643 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.362587 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47"} err="failed to get container status \"f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47\": rpc error: code = NotFound desc = could not find container \"f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47\": container with ID starting with f7ee279077c863a3da525c794812c14164d5b233b313784f67af7fdf1e189e47 not found: ID does not exist" May 29 01:11:50.373666 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.373643 2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c"] May 29 01:11:50.377163 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:50.377142 2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-scm6c"] May 29 01:11:52.141498 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:11:52.141468 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa9e73e4-3610-49d3-9b61-04d81bf88fb0" path="/var/lib/kubelet/pods/fa9e73e4-3610-49d3-9b61-04d81bf88fb0/volumes" May 29 01:12:22.163095 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:22.163068 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:12:22.166381 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:22.166361 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:12:55.037372 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.037282 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp"] May 29 01:12:55.037967 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.037845 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa9e73e4-3610-49d3-9b61-04d81bf88fb0" containerName="manager" May 29 01:12:55.037967 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.037864 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa9e73e4-3610-49d3-9b61-04d81bf88fb0" containerName="manager" May 29 01:12:55.038095 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.037989 2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="fa9e73e4-3610-49d3-9b61-04d81bf88fb0" containerName="manager" May 29 01:12:55.040958 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.040938 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.044228 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.044209 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f6qxk\"" May 29 01:12:55.051815 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.051796 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp"] May 29 01:12:55.112344 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.112319 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvbqc\" (UniqueName: \"kubernetes.io/projected/706aa08f-d808-416b-b67f-c23644c439b4-kube-api-access-vvbqc\") pod \"kuadrant-operator-controller-manager-55c7f4c975-m8gmp\" (UID: \"706aa08f-d808-416b-b67f-c23644c439b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.112454 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.112367 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/706aa08f-d808-416b-b67f-c23644c439b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-m8gmp\" (UID: \"706aa08f-d808-416b-b67f-c23644c439b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.213100 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.213076 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/706aa08f-d808-416b-b67f-c23644c439b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-m8gmp\" (UID: \"706aa08f-d808-416b-b67f-c23644c439b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.213307 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.213284 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vvbqc\" (UniqueName: \"kubernetes.io/projected/706aa08f-d808-416b-b67f-c23644c439b4-kube-api-access-vvbqc\") pod \"kuadrant-operator-controller-manager-55c7f4c975-m8gmp\" (UID: \"706aa08f-d808-416b-b67f-c23644c439b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.213462 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.213443 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/706aa08f-d808-416b-b67f-c23644c439b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-m8gmp\" (UID: \"706aa08f-d808-416b-b67f-c23644c439b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.221433 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.221412 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvbqc\" (UniqueName: \"kubernetes.io/projected/706aa08f-d808-416b-b67f-c23644c439b4-kube-api-access-vvbqc\") pod \"kuadrant-operator-controller-manager-55c7f4c975-m8gmp\" (UID: \"706aa08f-d808-416b-b67f-c23644c439b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.352355 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.352278 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.490113 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.490091 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp"] May 29 01:12:55.492237 ip-10-0-141-236 kubenswrapper[2551]: W0529 01:12:55.492207 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706aa08f_d808_416b_b67f_c23644c439b4.slice/crio-291f3da470b2eb6133f295bc2b1c89fafc50f41a1c88b1d990cf1b4f3a6b41e1 WatchSource:0}: Error finding container 291f3da470b2eb6133f295bc2b1c89fafc50f41a1c88b1d990cf1b4f3a6b41e1: Status 404 returned error can't find the container with id 291f3da470b2eb6133f295bc2b1c89fafc50f41a1c88b1d990cf1b4f3a6b41e1 May 29 01:12:55.494497 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.494476 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 01:12:55.593220 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.593192 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" event={"ID":"706aa08f-d808-416b-b67f-c23644c439b4","Type":"ContainerStarted","Data":"a92fe7d3e04b4ad7fbb23f3165dc6dec84bfcc22a8e9b32cec04863fe5425a6c"} May 29 01:12:55.593304 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.593227 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" event={"ID":"706aa08f-d808-416b-b67f-c23644c439b4","Type":"ContainerStarted","Data":"291f3da470b2eb6133f295bc2b1c89fafc50f41a1c88b1d990cf1b4f3a6b41e1"} May 29 01:12:55.593304 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.593302 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:12:55.615955 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:12:55.615883 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" podStartSLOduration=0.61586882 podStartE2EDuration="615.86882ms" podCreationTimestamp="2026-05-29 01:12:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 01:12:55.614434071 +0000 UTC m=+1534.074637523" watchObservedRunningTime="2026-05-29 01:12:55.61586882 +0000 UTC m=+1534.076072270" May 29 01:13:06.598942 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:13:06.598913 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-m8gmp" May 29 01:17:22.187989 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:17:22.187877 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:17:22.194738 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:17:22.191299 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:22:22.213523 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:22.213423 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:22:22.221170 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:22.221147 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:22:42.601487 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:42.601456 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-8srdn_efba5428-b3ed-4d0c-9b29-0c4929f74e65/manager/0.log" May 29 01:22:42.728593 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:42.728560 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-586b46cd66-q6m9v_343cd0c6-a66e-4621-9433-b0cc0d0c6ae8/maas-api/0.log" May 29 01:22:43.074571 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:43.074540 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b9fdf5dcf-xjdj5_7192fc54-a6c8-44f0-95c9-23203c5e435d/manager/0.log" May 29 01:22:43.196139 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:43.196113 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-bgw8q_20a81a50-d899-4b3a-9fee-db2f1da9ccde/manager/1.log" May 29 01:22:43.543687 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:43.543633 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d9b8687fd-xhn6d_2425a0be-7ed3-41c3-8052-e8ffa9d27924/manager/0.log" May 29 01:22:43.653801 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:43.653777 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-9q4bj_b3f0fc41-0785-4898-8852-6a3288befe97/postgres/0.log" May 29 01:22:45.020135 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:45.020082 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-rg7bm_1cdb5df8-9337-4ee0-82f6-364c86299e0c/manager/0.log" May 29 01:22:45.485050 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:45.485027 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-m8gmp_706aa08f-d808-416b-b67f-c23644c439b4/manager/0.log" May 29 01:22:46.183877 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:46.183845 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-xpx5j_7a585311-2208-477b-970f-086368c6e6aa/discovery/0.log" May 29 01:22:46.302016 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:46.301987 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7959ccfcc5-4sv64_387c1d2d-b813-4d44-b0cc-35ece0ebd1c3/kube-auth-proxy/0.log" May 29 01:22:46.751615 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:46.751586 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fc48bf698-5bp8x_8eaf00ba-f505-4f03-8dc0-599c0d80af9f/router/0.log" May 29 01:22:47.219019 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:47.218988 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6_03ca099b-6665-4598-8fed-cb9136ca343e/storage-initializer/0.log" May 29 01:22:47.227537 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:47.227514 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-8c9g6_03ca099b-6665-4598-8fed-cb9136ca343e/main/0.log" May 29 01:22:47.335923 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:47.335897 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-kkgn2_0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc/storage-initializer/0.log" May 29 01:22:47.342919 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:47.342892 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-kkgn2_0ee30f30-9d2e-4e84-a2e1-9f75859cf9bc/main/0.log" May 29 01:22:47.449082 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:47.449059 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d_78db6ad4-edef-4d40-ab94-1295d43ee7b2/storage-initializer/0.log" May 29 01:22:47.457934 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:47.457915 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmkj7d_78db6ad4-edef-4d40-ab94-1295d43ee7b2/main/0.log" May 29 01:22:54.926301 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:54.926271 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vgjjf_47a98844-8ffd-4487-b470-f3d17d86446d/global-pull-secret-syncer/0.log" May 29 01:22:54.975385 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:54.975361 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-979kv_94ea817f-e314-4f55-a064-dfafa9853ff7/konnectivity-agent/0.log" May 29 01:22:55.084829 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:55.084779 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-236.ec2.internal_b9682f407829b9ba775328d34f44bc9c/haproxy/0.log" May 29 01:22:59.414291 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:59.414257 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-rg7bm_1cdb5df8-9337-4ee0-82f6-364c86299e0c/manager/0.log" May 29 01:22:59.588411 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:22:59.588387 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-m8gmp_706aa08f-d808-416b-b67f-c23644c439b4/manager/0.log" May 29 01:23:01.385531 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.385500 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tgcvp_764832ca-7360-4c3a-96a3-057d82c23654/node-exporter/0.log" May 29 01:23:01.399966 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.399941 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tgcvp_764832ca-7360-4c3a-96a3-057d82c23654/kube-rbac-proxy/0.log" May 29 01:23:01.415162 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.415144 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tgcvp_764832ca-7360-4c3a-96a3-057d82c23654/init-textfile/0.log" May 29 01:23:01.630915 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.630895 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-fmsxj_35f5edeb-8f82-4972-8a38-b39c237b60c4/prometheus-operator/0.log" May 29 01:23:01.643958 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.643902 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-fmsxj_35f5edeb-8f82-4972-8a38-b39c237b60c4/kube-rbac-proxy/0.log" May 29 01:23:01.692307 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.692287 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cb7d9d955-fw67s_b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0/telemeter-client/0.log" May 29 01:23:01.707398 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.707380 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cb7d9d955-fw67s_b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0/reload/0.log" May 29 01:23:01.724581 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.724557 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cb7d9d955-fw67s_b7e3ebb6-faba-4aae-a9cc-1786cf72c2b0/kube-rbac-proxy/0.log" May 29 01:23:01.752334 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.752307 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55d6f9fc57-khts4_96d65807-0a2f-4ecc-a509-454fc349c6de/thanos-query/0.log" May 29 01:23:01.770625 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.770609 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55d6f9fc57-khts4_96d65807-0a2f-4ecc-a509-454fc349c6de/kube-rbac-proxy-web/0.log" May 29 01:23:01.787075 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.787052 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55d6f9fc57-khts4_96d65807-0a2f-4ecc-a509-454fc349c6de/kube-rbac-proxy/0.log" May 29 01:23:01.809657 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.809637 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55d6f9fc57-khts4_96d65807-0a2f-4ecc-a509-454fc349c6de/prom-label-proxy/0.log" May 29 01:23:01.825030 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.825015 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55d6f9fc57-khts4_96d65807-0a2f-4ecc-a509-454fc349c6de/kube-rbac-proxy-rules/0.log" May 29 01:23:01.849632 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:01.849610 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55d6f9fc57-khts4_96d65807-0a2f-4ecc-a509-454fc349c6de/kube-rbac-proxy-metrics/0.log" May 29 01:23:03.442935 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.442905 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/2.log" May 29 01:23:03.450446 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.450415 2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8"] May 29 01:23:03.451032 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.450986 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-m6z2w_bdeff395-20f1-4a4e-8e7a-f0bc2f5feac2/console-operator/3.log" May 29 01:23:03.454490 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.454472 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.457300 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.457282 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-979vp\"/\"kube-root-ca.crt\"" May 29 01:23:03.458703 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.458684 2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-979vp\"/\"default-dockercfg-pvr4b\"" May 29 01:23:03.458910 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.458739 2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-979vp\"/\"openshift-service-ca.crt\"" May 29 01:23:03.461493 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.461468 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8"] May 29 01:23:03.583196 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.583169 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvq6s\" (UniqueName: \"kubernetes.io/projected/d7e26ee8-176d-4f4f-9c25-5d91fc598076-kube-api-access-kvq6s\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.583299 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.583209 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-sys\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.583299 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.583250 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-podres\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.583413 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.583298 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-proc\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.583413 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.583352 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-lib-modules\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684609 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684582 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvq6s\" (UniqueName: \"kubernetes.io/projected/d7e26ee8-176d-4f4f-9c25-5d91fc598076-kube-api-access-kvq6s\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684714 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684615 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-sys\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684714 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684642 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-podres\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684714 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684695 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-proc\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684714 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684712 2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-lib-modules\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684893 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684721 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-sys\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684893 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684820 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-proc\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684893 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684869 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-podres\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.684893 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.684868 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7e26ee8-176d-4f4f-9c25-5d91fc598076-lib-modules\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.694030 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.693979 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvq6s\" (UniqueName: \"kubernetes.io/projected/d7e26ee8-176d-4f4f-9c25-5d91fc598076-kube-api-access-kvq6s\") pod \"perf-node-gather-daemonset-p9tn8\" (UID: \"d7e26ee8-176d-4f4f-9c25-5d91fc598076\") " pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.765649 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.765626 2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:03.901537 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.901511 2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8"] May 29 01:23:03.904236 ip-10-0-141-236 kubenswrapper[2551]: W0529 01:23:03.904209 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd7e26ee8_176d_4f4f_9c25_5d91fc598076.slice/crio-feafd6159e57fd6f31f199570928a3a2f0d43b83efce57b9831dc4498f2978f7 WatchSource:0}: Error finding container feafd6159e57fd6f31f199570928a3a2f0d43b83efce57b9831dc4498f2978f7: Status 404 returned error can't find the container with id feafd6159e57fd6f31f199570928a3a2f0d43b83efce57b9831dc4498f2978f7 May 29 01:23:03.909137 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.909119 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 29 01:23:03.950078 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.950056 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bd5d89bf9-mxxbv_6f082769-70a1-4fb6-9283-bfabdc28318a/console/0.log" May 29 01:23:03.982716 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:03.982694 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-fzcdb_9ecf3dfd-1802-4256-bc32-2b155f7ff18b/download-server/0.log" May 29 01:23:04.430537 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:04.430205 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-rhqlg_9cb49a07-dae6-457b-bd3b-a054879e505c/volume-data-source-validator/0.log" May 29 01:23:04.802314 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:04.802281 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" event={"ID":"d7e26ee8-176d-4f4f-9c25-5d91fc598076","Type":"ContainerStarted","Data":"3a820e432b731fbd24d02088015920b912733b8c27e8be49e4c73067a36bd54e"} May 29 01:23:04.802314 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:04.802316 2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" event={"ID":"d7e26ee8-176d-4f4f-9c25-5d91fc598076","Type":"ContainerStarted","Data":"feafd6159e57fd6f31f199570928a3a2f0d43b83efce57b9831dc4498f2978f7"} May 29 01:23:04.802693 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:04.802378 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:04.820925 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:04.820879 2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" podStartSLOduration=1.820869029 podStartE2EDuration="1.820869029s" podCreationTimestamp="2026-05-29 01:23:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 01:23:04.819852908 +0000 UTC m=+2143.280056358" watchObservedRunningTime="2026-05-29 01:23:04.820869029 +0000 UTC m=+2143.281072479" May 29 01:23:05.304403 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:05.304367 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n5lpq_a0aa4161-0aa4-4e61-b55e-d737c3bfaecd/dns/0.log" May 29 01:23:05.320619 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:05.320596 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n5lpq_a0aa4161-0aa4-4e61-b55e-d737c3bfaecd/kube-rbac-proxy/0.log" May 29 01:23:05.338463 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:05.338401 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4tt5k_31951da8-a753-4b89-9838-3f0c2c5505d1/dns-node-resolver/0.log" May 29 01:23:05.871047 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:05.871015 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2zcx5_f0790f93-cef8-451c-8dc7-358ce10becf4/node-ca/0.log" May 29 01:23:06.809266 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:06.809234 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-xpx5j_7a585311-2208-477b-970f-086368c6e6aa/discovery/0.log" May 29 01:23:06.827800 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:06.827733 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7959ccfcc5-4sv64_387c1d2d-b813-4d44-b0cc-35ece0ebd1c3/kube-auth-proxy/0.log" May 29 01:23:07.012325 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:07.012300 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fc48bf698-5bp8x_8eaf00ba-f505-4f03-8dc0-599c0d80af9f/router/0.log" May 29 01:23:07.536736 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:07.536711 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n5szr_bec90e87-7a28-4f14-83c6-61def84dbed9/serve-healthcheck-canary/0.log" May 29 01:23:08.113014 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:08.112983 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rn29k_bdd6492a-262d-4198-8b94-a212d75eba2e/kube-rbac-proxy/0.log" May 29 01:23:08.130449 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:08.130425 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rn29k_bdd6492a-262d-4198-8b94-a212d75eba2e/exporter/0.log" May 29 01:23:08.147716 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:08.147689 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rn29k_bdd6492a-262d-4198-8b94-a212d75eba2e/extractor/0.log" May 29 01:23:09.978786 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:09.978744 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-8srdn_efba5428-b3ed-4d0c-9b29-0c4929f74e65/manager/0.log" May 29 01:23:10.028344 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:10.028321 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-586b46cd66-q6m9v_343cd0c6-a66e-4621-9433-b0cc0d0c6ae8/maas-api/0.log" May 29 01:23:10.179435 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:10.179399 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b9fdf5dcf-xjdj5_7192fc54-a6c8-44f0-95c9-23203c5e435d/manager/0.log" May 29 01:23:10.195711 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:10.195687 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-bgw8q_20a81a50-d899-4b3a-9fee-db2f1da9ccde/manager/0.log" May 29 01:23:10.216403 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:10.216383 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-bgw8q_20a81a50-d899-4b3a-9fee-db2f1da9ccde/manager/1.log" May 29 01:23:10.306551 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:10.306464 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d9b8687fd-xhn6d_2425a0be-7ed3-41c3-8052-e8ffa9d27924/manager/0.log" May 29 01:23:10.324023 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:10.323994 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-9q4bj_b3f0fc41-0785-4898-8852-6a3288befe97/postgres/0.log" May 29 01:23:10.817243 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:10.817222 2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-979vp/perf-node-gather-daemonset-p9tn8" May 29 01:23:15.712126 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:15.712095 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-w99v9_972f368e-2614-4c0e-8bae-5c008540add4/migrator/0.log" May 29 01:23:15.727404 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:15.727378 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-w99v9_972f368e-2614-4c0e-8bae-5c008540add4/graceful-termination/0.log" May 29 01:23:17.118725 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.118696 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4pr7f_f83593a1-25ed-4d6a-94f6-05e427aea6eb/kube-multus-additional-cni-plugins/0.log" May 29 01:23:17.139493 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.139469 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4pr7f_f83593a1-25ed-4d6a-94f6-05e427aea6eb/egress-router-binary-copy/0.log" May 29 01:23:17.156478 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.156454 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4pr7f_f83593a1-25ed-4d6a-94f6-05e427aea6eb/cni-plugins/0.log" May 29 01:23:17.182948 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.182919 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4pr7f_f83593a1-25ed-4d6a-94f6-05e427aea6eb/bond-cni-plugin/0.log" May 29 01:23:17.206074 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.206040 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4pr7f_f83593a1-25ed-4d6a-94f6-05e427aea6eb/routeoverride-cni/0.log" May 29 01:23:17.224262 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.224243 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4pr7f_f83593a1-25ed-4d6a-94f6-05e427aea6eb/whereabouts-cni-bincopy/0.log" May 29 01:23:17.244006 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.243984 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4pr7f_f83593a1-25ed-4d6a-94f6-05e427aea6eb/whereabouts-cni/0.log" May 29 01:23:17.556570 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.556547 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jjscg_cec494db-636c-4310-9706-a9ec296896ff/kube-multus/0.log" May 29 01:23:17.640289 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.640263 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-px5qc_12335c23-f1a8-4eae-9210-8801fa6ce0bd/network-metrics-daemon/0.log" May 29 01:23:17.653446 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:17.653421 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-px5qc_12335c23-f1a8-4eae-9210-8801fa6ce0bd/kube-rbac-proxy/0.log" May 29 01:23:18.788631 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:18.788601 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4vn9b_dfdb2801-12f3-4d43-a796-c512dd303655/ovn-controller/0.log" May 29 01:23:18.824352 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:18.824313 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4vn9b_dfdb2801-12f3-4d43-a796-c512dd303655/ovn-acl-logging/0.log" May 29 01:23:18.843924 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:18.843904 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4vn9b_dfdb2801-12f3-4d43-a796-c512dd303655/kube-rbac-proxy-node/0.log" May 29 01:23:18.859831 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:18.859811 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4vn9b_dfdb2801-12f3-4d43-a796-c512dd303655/kube-rbac-proxy-ovn-metrics/0.log" May 29 01:23:18.874041 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:18.874021 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4vn9b_dfdb2801-12f3-4d43-a796-c512dd303655/northd/0.log" May 29 01:23:18.890488 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:18.890466 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4vn9b_dfdb2801-12f3-4d43-a796-c512dd303655/nbdb/0.log" May 29 01:23:18.906044 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:18.906024 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4vn9b_dfdb2801-12f3-4d43-a796-c512dd303655/sbdb/0.log" May 29 01:23:19.082477 ip-10-0-141-236 kubenswrapper[2551]: I0529 01:23:19.082403 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4vn9b_dfdb2801-12f3-4d43-a796-c512dd303655/ovnkube-controller/0.log"