Jun 03 16:18:17.567822 ip-10-0-128-141 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 03 16:18:17.567835 ip-10-0-128-141 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 03 16:18:17.567844 ip-10-0-128-141 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 03 16:18:17.568061 ip-10-0-128-141 systemd[1]: Failed to start Kubernetes Kubelet. Jun 03 16:18:27.577488 ip-10-0-128-141 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 03 16:18:27.577508 ip-10-0-128-141 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f614bce098154395ace2c699e95f8e84 -- Jun 03 16:21:09.619643 ip-10-0-128-141 systemd[1]: Starting Kubernetes Kubelet... Jun 03 16:21:10.060012 ip-10-0-128-141 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:21:10.060012 ip-10-0-128-141 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 16:21:10.060012 ip-10-0-128-141 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:21:10.060012 ip-10-0-128-141 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 16:21:10.060558 ip-10-0-128-141 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:21:10.061718 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.061651 2576 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 16:21:10.065805 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065791 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:21:10.065805 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065805 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065809 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065812 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065815 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065818 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065821 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065824 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065827 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065830 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065833 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065835 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065839 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065842 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065845 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065848 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065850 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065853 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065855 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065858 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065861 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:21:10.065869 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065863 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065866 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065869 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065871 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065875 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065880 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065883 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065885 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065888 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065890 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065893 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065896 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065899 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065902 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065904 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065907 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065909 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065912 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065914 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:21:10.066342 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065917 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065919 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065922 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065924 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065926 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065930 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065933 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065936 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065939 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065942 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065944 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065947 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065949 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065952 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065954 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065957 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065960 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065963 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065965 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:21:10.066800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065968 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065970 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065974 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065977 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065980 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065982 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065984 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065987 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065990 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065992 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065995 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065997 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.065999 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066002 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066004 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066007 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066009 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066012 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066014 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066017 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:21:10.067240 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066019 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066022 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066024 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066027 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066029 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066032 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066034 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066379 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066386 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066390 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066394 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066397 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066400 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066403 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066405 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066408 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066410 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066413 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066415 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066418 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:21:10.067735 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066421 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066423 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066426 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066428 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066431 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066433 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066436 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066438 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066440 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066443 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066445 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066447 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066450 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066452 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066455 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066457 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066460 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066462 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066465 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:21:10.068214 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066468 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066471 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066474 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066476 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066479 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066481 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066483 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066486 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066488 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066491 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066493 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066495 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066498 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066501 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066503 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066505 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066508 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066510 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066513 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:21:10.068695 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066515 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066517 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066520 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066522 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066525 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066527 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066530 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066532 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066535 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066537 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066540 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066542 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066544 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066547 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066550 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066553 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066556 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066558 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066560 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066563 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:21:10.069149 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066565 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066567 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066571 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066573 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066576 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066578 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066581 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066591 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066594 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066596 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066600 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066603 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066605 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066608 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.066610 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067902 2576 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067911 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067916 2576 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067920 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067924 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067927 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 16:21:10.069653 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067931 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067935 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067938 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067941 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067944 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067948 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067951 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067954 2576 flags.go:64] FLAG: --cgroup-root="" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067957 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067959 2576 flags.go:64] FLAG: --client-ca-file="" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067962 2576 flags.go:64] FLAG: --cloud-config="" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067965 2576 flags.go:64] FLAG: --cloud-provider="external" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067967 2576 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067972 2576 flags.go:64] FLAG: --cluster-domain="" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067975 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067978 2576 flags.go:64] FLAG: --config-dir="" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067981 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067984 2576 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067987 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067991 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067994 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067997 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.067999 2576 flags.go:64] FLAG: --contention-profiling="false" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068003 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 16:21:10.070205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068006 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068009 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068011 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068015 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068018 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068020 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068024 2576 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068026 2576 flags.go:64] FLAG: --enable-server="true" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068029 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068033 2576 flags.go:64] FLAG: --event-burst="100" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068036 2576 flags.go:64] FLAG: --event-qps="50" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068039 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068042 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068045 2576 flags.go:64] FLAG: --eviction-hard="" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068048 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068051 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068054 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068057 2576 flags.go:64] FLAG: --eviction-soft="" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068060 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068062 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068065 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068068 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068071 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068074 2576 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068076 2576 flags.go:64] FLAG: --feature-gates="" Jun 03 16:21:10.070853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068080 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068082 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068085 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068089 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068092 2576 flags.go:64] FLAG: --healthz-port="10248" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068095 2576 flags.go:64] FLAG: --help="false" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068097 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068100 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068103 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068106 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068109 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068112 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068115 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068118 2576 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068120 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068123 2576 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068125 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068129 2576 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068132 2576 flags.go:64] FLAG: --kube-reserved="" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068135 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068138 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068141 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068143 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068146 2576 flags.go:64] FLAG: --lock-file="" Jun 03 16:21:10.071579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068149 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068152 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068154 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068159 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068162 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068165 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068167 2576 flags.go:64] FLAG: --logging-format="text" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068170 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068173 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068175 2576 flags.go:64] FLAG: --manifest-url="" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068178 2576 flags.go:64] FLAG: --manifest-url-header="" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068182 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068185 2576 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068189 2576 flags.go:64] FLAG: --max-pods="110" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068192 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068195 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068198 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068200 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068203 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068206 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068209 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068216 2576 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068219 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068221 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 16:21:10.072197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068224 2576 flags.go:64] FLAG: --pod-cidr="" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068228 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068239 2576 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068241 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068244 2576 flags.go:64] FLAG: --pods-per-core="0" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068247 2576 flags.go:64] FLAG: --port="10250" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068250 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068253 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00864a7e0820e839e" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068256 2576 flags.go:64] FLAG: --qos-reserved="" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068259 2576 flags.go:64] FLAG: --read-only-port="10255" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068262 2576 flags.go:64] FLAG: --register-node="true" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068264 2576 flags.go:64] FLAG: --register-schedulable="true" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068267 2576 flags.go:64] FLAG: --register-with-taints="" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068274 2576 flags.go:64] FLAG: --registry-burst="10" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068277 2576 flags.go:64] FLAG: --registry-qps="5" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068279 2576 flags.go:64] FLAG: --reserved-cpus="" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068282 2576 flags.go:64] FLAG: --reserved-memory="" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068286 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068289 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068291 2576 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068294 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068297 2576 flags.go:64] FLAG: --runonce="false" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068300 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068303 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068306 2576 flags.go:64] FLAG: --seccomp-default="false" Jun 03 16:21:10.072881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068309 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068311 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068314 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068333 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068337 2576 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068339 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068342 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068345 2576 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068348 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068351 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068354 2576 flags.go:64] FLAG: --system-cgroups="" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068356 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068361 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068364 2576 flags.go:64] FLAG: --tls-cert-file="" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068367 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068371 2576 flags.go:64] FLAG: --tls-min-version="" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068374 2576 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068377 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068380 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068383 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068386 2576 flags.go:64] FLAG: --v="2" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068390 2576 flags.go:64] FLAG: --version="false" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068394 2576 flags.go:64] FLAG: --vmodule="" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068397 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.068400 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 16:21:10.073555 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068486 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068489 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068492 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068495 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068498 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068501 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068504 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068507 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068509 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068512 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068515 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068518 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068520 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068523 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068525 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068528 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068530 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068533 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068535 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:21:10.074203 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068537 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068540 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068542 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068545 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068547 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068549 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068552 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068554 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068557 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068559 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068562 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068564 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068567 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068569 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068572 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068574 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068577 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068579 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068585 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068587 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:21:10.074699 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068590 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068592 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068594 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068597 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068600 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068602 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068605 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068607 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068611 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068616 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068619 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068622 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068625 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068629 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068632 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068634 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068637 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068639 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068641 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:21:10.075213 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068644 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068649 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068651 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068654 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068656 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068659 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068661 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068664 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068666 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068669 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068671 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068674 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068676 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068679 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068681 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068684 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068686 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068689 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068691 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068694 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:21:10.075688 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068696 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068698 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068703 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068706 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068708 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068711 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068713 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.068715 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.069405 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.075346 2576 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.075361 2576 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075404 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075424 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075429 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075432 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075436 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:21:10.076163 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075438 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075441 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075444 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075447 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075449 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075452 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075455 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075458 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075460 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075463 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075466 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075476 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075480 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075483 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075485 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075488 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075491 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075494 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075496 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075499 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:21:10.076575 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075501 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075504 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075506 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075509 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075511 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075514 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075518 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075521 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075524 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075526 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075529 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075531 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075534 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075536 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075540 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075543 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075547 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075549 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075552 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:21:10.077050 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075554 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075557 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075559 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075562 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075565 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075567 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075570 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075573 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075575 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075578 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075580 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075583 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075585 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075588 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075590 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075593 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075596 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075599 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075601 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075604 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:21:10.077618 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075607 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075609 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075612 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075614 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075617 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075619 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075622 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075624 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075626 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075629 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075631 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075634 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075637 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075639 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075641 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075644 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075647 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075650 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075652 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075654 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:21:10.078099 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075657 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075660 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.075666 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075752 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075757 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075761 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075764 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075767 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075769 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075772 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075774 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075777 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075780 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075783 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075785 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:21:10.078582 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075788 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075790 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075793 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075795 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075798 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075800 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075803 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075805 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075808 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075810 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075812 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075815 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075818 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075820 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075823 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075825 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075828 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075830 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075833 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075835 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:21:10.078954 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075837 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075840 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075842 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075844 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075847 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075849 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075851 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075854 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075857 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075860 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075862 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075865 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075867 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075870 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075872 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075874 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075877 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075879 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075881 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075884 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:21:10.079431 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075886 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075889 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075891 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075893 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075896 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075898 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075901 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075903 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075906 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075908 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075911 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075913 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075915 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075917 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075920 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075922 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075925 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075927 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075930 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:21:10.079923 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075933 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075935 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075937 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075940 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075942 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075945 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075947 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075950 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075952 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075956 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075959 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075962 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075964 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075966 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:10.075969 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:21:10.080381 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.075973 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:21:10.080745 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.076622 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 16:21:10.080745 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.078441 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 16:21:10.080745 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.079658 2576 server.go:1019] "Starting client certificate rotation" Jun 03 16:21:10.080745 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.079751 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 16:21:10.080745 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.080556 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 16:21:10.109986 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.109966 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 16:21:10.111684 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.111668 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 16:21:10.124550 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.124531 2576 log.go:25] "Validated CRI v1 runtime API" Jun 03 16:21:10.130086 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.130072 2576 log.go:25] "Validated CRI v1 image API" Jun 03 16:21:10.131252 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.131234 2576 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 16:21:10.133369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.133351 2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ba3f7207-3a32-40af-a59b-82c2f7769613:/dev/nvme0n1p3 d7e81708-b012-4d2a-9a93-4a40614af5b9:/dev/nvme0n1p4] Jun 03 16:21:10.133417 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.133368 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 16:21:10.137509 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.137407 2576 manager.go:217] Machine: {Timestamp:2026-06-03 16:21:10.136873638 +0000 UTC m=+0.400937504 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3201812 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec269cc750fe41022740fa8fcab502b1 SystemUUID:ec269cc7-50fe-4102-2740-fa8fcab502b1 BootID:f614bce0-9815-4395-ace2-c699e95f8e84 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:be:4c:81:03:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:be:4c:81:03:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:a1:55:bc:f3:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 16:21:10.137509 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.137502 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 16:21:10.137671 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.137564 2576 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 16:21:10.140373 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.140355 2576 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 16:21:10.142056 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.142032 2576 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 16:21:10.142173 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.142058 2576 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 16:21:10.142223 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.142181 2576 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 16:21:10.142223 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.142190 2576 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 16:21:10.142223 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.142197 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 16:21:10.142223 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.142209 2576 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 16:21:10.143693 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.143682 2576 state_mem.go:36] "Initialized new in-memory state store" Jun 03 16:21:10.143796 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.143787 2576 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 16:21:10.146034 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.146024 2576 kubelet.go:491] "Attempting to sync node with API server" Jun 03 16:21:10.146064 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.146037 2576 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 16:21:10.146872 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.146863 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 16:21:10.146911 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.146876 2576 kubelet.go:397] "Adding apiserver pod source" Jun 03 16:21:10.146911 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.146884 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 16:21:10.147925 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.147911 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 16:21:10.148005 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.147930 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 16:21:10.151393 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.151378 2576 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 16:21:10.153233 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.153210 2576 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 16:21:10.155443 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155428 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 16:21:10.155525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155448 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 16:21:10.155525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155458 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 16:21:10.155525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155472 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 16:21:10.155525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155485 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 16:21:10.155525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155493 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 16:21:10.155525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155515 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 16:21:10.155525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155523 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 16:21:10.155754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155534 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 16:21:10.155754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155542 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 16:21:10.155754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155560 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 16:21:10.155754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.155569 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 16:21:10.156508 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.156493 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 16:21:10.156508 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.156510 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 16:21:10.156654 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.156626 2576 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 16:21:10.156708 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.156689 2576 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 16:21:10.159285 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.159271 2576 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 16:21:10.159365 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.159306 2576 server.go:1295] "Started kubelet" Jun 03 16:21:10.159406 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.159377 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 16:21:10.159941 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.159918 2576 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 03 16:21:10.159941 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.159902 2576 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 16:21:10.160051 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.159962 2576 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 16:21:10.159951 ip-10-0-128-141 systemd[1]: Started Kubernetes Kubelet. Jun 03 16:21:10.161251 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.161234 2576 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 16:21:10.161998 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.161981 2576 server.go:317] "Adding debug handlers to kubelet server" Jun 03 16:21:10.165656 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.165641 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 16:21:10.166088 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166073 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 16:21:10.166150 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166142 2576 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 16:21:10.166197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166153 2576 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 16:21:10.166197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166169 2576 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 16:21:10.166269 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166243 2576 reconstruct.go:97] "Volume reconstruction finished" Jun 03 16:21:10.166269 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166251 2576 reconciler.go:26] "Reconciler: start to sync state" Jun 03 16:21:10.166744 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166727 2576 factory.go:55] Registering systemd factory Jun 03 16:21:10.166744 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166745 2576 factory.go:223] Registration of the systemd container factory successfully Jun 03 16:21:10.166963 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.166938 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.167046 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166968 2576 factory.go:153] Registering CRI-O factory Jun 03 16:21:10.167046 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.166982 2576 factory.go:223] Registration of the crio container factory successfully Jun 03 16:21:10.167046 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.167022 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 16:21:10.167046 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.167038 2576 factory.go:103] Registering Raw factory Jun 03 16:21:10.167203 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.167051 2576 manager.go:1196] Started watching for new ooms in manager Jun 03 16:21:10.167563 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.167549 2576 manager.go:319] Starting recovery of all containers Jun 03 16:21:10.167666 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.167599 2576 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 16:21:10.168698 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.168676 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fvqhk" Jun 03 16:21:10.175600 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.175572 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 03 16:21:10.175600 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.175593 2576 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 03 16:21:10.176732 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.176549 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fvqhk" Jun 03 16:21:10.176852 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.175786 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-141.ec2.internal.18b59ea78d1f99ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-141.ec2.internal,UID:ip-10-0-128-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-141.ec2.internal,},FirstTimestamp:2026-06-03 16:21:10.159284686 +0000 UTC m=+0.423348552,LastTimestamp:2026-06-03 16:21:10.159284686 +0000 UTC m=+0.423348552,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-141.ec2.internal,}" Jun 03 16:21:10.177603 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.177586 2576 manager.go:324] Recovery completed Jun 03 16:21:10.181701 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.181686 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:21:10.183334 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.183309 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:21:10.183406 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.183348 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:21:10.183406 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.183381 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:21:10.183862 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.183849 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 16:21:10.183862 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.183862 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 16:21:10.183949 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.183877 2576 state_mem.go:36] "Initialized new in-memory state store" Jun 03 16:21:10.185192 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.185140 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-141.ec2.internal.18b59ea78e8e918a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-141.ec2.internal,UID:ip-10-0-128-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-141.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-141.ec2.internal,},FirstTimestamp:2026-06-03 16:21:10.183334282 +0000 UTC m=+0.447398152,LastTimestamp:2026-06-03 16:21:10.183334282 +0000 UTC m=+0.447398152,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-141.ec2.internal,}" Jun 03 16:21:10.186541 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.186470 2576 policy_none.go:49] "None policy: Start" Jun 03 16:21:10.186541 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.186483 2576 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 16:21:10.186541 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.186492 2576 state_mem.go:35] "Initializing new in-memory state store" Jun 03 16:21:10.220914 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.220901 2576 manager.go:341] "Starting Device Plugin manager" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.220931 2576 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.220940 2576 server.go:85] "Starting device plugin registration server" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.221140 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.221149 2576 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.221228 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.221297 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.221306 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.221897 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 16:21:10.236473 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.221936 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.307133 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.307115 2576 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 16:21:10.308232 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.308216 2576 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 16:21:10.308336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.308239 2576 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 16:21:10.308336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.308253 2576 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 16:21:10.308336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.308259 2576 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 16:21:10.308336 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.308296 2576 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 16:21:10.310660 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.310611 2576 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:21:10.321424 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.321412 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:21:10.322074 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.322060 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:21:10.322131 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.322084 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:21:10.322131 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.322094 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:21:10.322131 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.322113 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.331198 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.331178 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.331198 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.331196 2576 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-141.ec2.internal\": node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.353392 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.353375 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.408878 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.408857 2576 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal"] Jun 03 16:21:10.408942 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.408915 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:21:10.410564 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.410550 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:21:10.410633 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.410576 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:21:10.410633 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.410589 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:21:10.411751 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.411737 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:21:10.411891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.411877 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.411933 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.411905 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:21:10.412349 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.412335 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:21:10.412440 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.412363 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:21:10.412440 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.412336 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:21:10.412440 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.412404 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:21:10.412440 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.412378 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:21:10.412440 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.412422 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:21:10.413605 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.413589 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.413681 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.413624 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:21:10.414198 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.414186 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:21:10.414254 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.414207 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:21:10.414254 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.414219 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:21:10.438752 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.438730 2576 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-141.ec2.internal\" not found" node="ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.442603 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.442589 2576 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-141.ec2.internal\" not found" node="ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.453864 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.453850 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.467650 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.467631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b690a59b3980ddc7968d917c387fd722-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"b690a59b3980ddc7968d917c387fd722\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.467717 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.467652 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b690a59b3980ddc7968d917c387fd722-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"b690a59b3980ddc7968d917c387fd722\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.467717 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.467671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167b61c3f33681bb1d2fd6165981fb2f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-141.ec2.internal\" (UID: \"167b61c3f33681bb1d2fd6165981fb2f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.554151 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.554122 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.568457 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.568410 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b690a59b3980ddc7968d917c387fd722-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"b690a59b3980ddc7968d917c387fd722\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.568529 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.568470 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b690a59b3980ddc7968d917c387fd722-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"b690a59b3980ddc7968d917c387fd722\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.568529 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.568492 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b690a59b3980ddc7968d917c387fd722-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"b690a59b3980ddc7968d917c387fd722\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.568529 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.568497 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b690a59b3980ddc7968d917c387fd722-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal\" (UID: \"b690a59b3980ddc7968d917c387fd722\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.568626 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.568539 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167b61c3f33681bb1d2fd6165981fb2f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-141.ec2.internal\" (UID: \"167b61c3f33681bb1d2fd6165981fb2f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.568626 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.568564 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167b61c3f33681bb1d2fd6165981fb2f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-141.ec2.internal\" (UID: \"167b61c3f33681bb1d2fd6165981fb2f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.654829 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.654799 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.741268 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.741252 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.745773 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:10.745757 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Jun 03 16:21:10.755442 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.755423 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.855954 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.855905 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:10.956364 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:10.956345 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:11.056822 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:11.056800 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:11.080224 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.080202 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 16:21:11.080656 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.080331 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 16:21:11.135349 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.134113 2576 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:21:11.156906 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:11.156868 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:11.166197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.166178 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 16:21:11.176550 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.176532 2576 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 16:21:11.178434 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.178411 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 16:16:10 +0000 UTC" deadline="2027-12-18 19:51:05.858647459 +0000 UTC" Jun 03 16:21:11.178490 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.178434 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13515h29m54.680216235s" Jun 03 16:21:11.197997 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.197980 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ftzwt" Jun 03 16:21:11.206075 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.206056 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ftzwt" Jun 03 16:21:11.257389 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:11.257362 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:11.303160 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:11.303127 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167b61c3f33681bb1d2fd6165981fb2f.slice/crio-4eaa49edcb14a2117916bb08e678cd209f261ef4c7e048fd6c7e1c30315e0051 WatchSource:0}: Error finding container 4eaa49edcb14a2117916bb08e678cd209f261ef4c7e048fd6c7e1c30315e0051: Status 404 returned error can't find the container with id 4eaa49edcb14a2117916bb08e678cd209f261ef4c7e048fd6c7e1c30315e0051 Jun 03 16:21:11.303401 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:11.303367 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb690a59b3980ddc7968d917c387fd722.slice/crio-92d527dc314614e356396c25ca4e2b37e690d77766d65b0c5e9f4d31e2c1db79 WatchSource:0}: Error finding container 92d527dc314614e356396c25ca4e2b37e690d77766d65b0c5e9f4d31e2c1db79: Status 404 returned error can't find the container with id 92d527dc314614e356396c25ca4e2b37e690d77766d65b0c5e9f4d31e2c1db79 Jun 03 16:21:11.306019 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.306005 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:21:11.311161 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.311120 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" event={"ID":"167b61c3f33681bb1d2fd6165981fb2f","Type":"ContainerStarted","Data":"4eaa49edcb14a2117916bb08e678cd209f261ef4c7e048fd6c7e1c30315e0051"} Jun 03 16:21:11.312072 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.312052 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" event={"ID":"b690a59b3980ddc7968d917c387fd722","Type":"ContainerStarted","Data":"92d527dc314614e356396c25ca4e2b37e690d77766d65b0c5e9f4d31e2c1db79"} Jun 03 16:21:11.358114 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:11.358093 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:11.453934 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.453867 2576 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:21:11.458866 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:11.458849 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-141.ec2.internal\" not found" Jun 03 16:21:11.468865 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.468844 2576 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:21:11.566880 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.566863 2576 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" Jun 03 16:21:11.578890 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.578873 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 16:21:11.579845 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.579833 2576 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" Jun 03 16:21:11.588681 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:11.588661 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 16:21:12.147876 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.147846 2576 apiserver.go:52] "Watching apiserver" Jun 03 16:21:12.154377 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.154349 2576 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 16:21:12.156913 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.156882 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx","openshift-dns/node-resolver-pndzh","openshift-multus/multus-additional-cni-plugins-jjcb8","openshift-multus/network-metrics-daemon-89gzm","openshift-network-diagnostics/network-check-target-2jjqb","kube-system/konnectivity-agent-j5x6b","openshift-cluster-node-tuning-operator/tuned-fs6bm","openshift-image-registry/node-ca-v7vzl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal","openshift-multus/multus-cmrwn","openshift-network-operator/iptables-alerter-cmrvv","openshift-ovn-kubernetes/ovnkube-node-68jfx","kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal"] Jun 03 16:21:12.158744 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.158722 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.160966 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.160946 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.161605 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.161455 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:21:12.161605 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.161513 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 16:21:12.161778 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.161606 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j2vdm\"" Jun 03 16:21:12.162414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.162245 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.163012 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.162995 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 16:21:12.163168 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.163153 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 16:21:12.163247 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.163230 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xq8pk\"" Jun 03 16:21:12.163764 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.163541 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:12.163764 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.163594 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:12.163764 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.163601 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:12.163764 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.163632 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:12.164867 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.164850 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 16:21:12.164995 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.164972 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 16:21:12.165092 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.165067 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 16:21:12.165152 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.165121 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vcnsx\"" Jun 03 16:21:12.165207 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.165154 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 16:21:12.165207 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.165175 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 16:21:12.166709 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.166509 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.168148 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.168129 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.168580 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.168561 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 16:21:12.169016 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.168998 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 16:21:12.169259 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.169243 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 16:21:12.169370 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.169315 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mrq5c\"" Jun 03 16:21:12.169693 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.169676 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:12.170194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.170176 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 16:21:12.170253 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.170235 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xd9cj\"" Jun 03 16:21:12.170523 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.170507 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 16:21:12.171595 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.171142 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 16:21:12.171687 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.171617 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.174115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.171928 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.174115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.173032 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 16:21:12.174115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.173360 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-64pbx\"" Jun 03 16:21:12.174115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.173390 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 16:21:12.174115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.173903 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.174775 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.174556 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 16:21:12.175913 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.175870 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ksmcl\"" Jun 03 16:21:12.175999 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.175913 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:21:12.176048 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.175870 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 16:21:12.176259 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176242 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6sbh7\"" Jun 03 16:21:12.176352 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176276 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 16:21:12.176611 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176594 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-system-cni-dir\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.176699 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176623 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50816dc4-c2f9-4346-b8fa-d8e493440d5b-konnectivity-ca\") pod \"konnectivity-agent-j5x6b\" (UID: \"50816dc4-c2f9-4346-b8fa-d8e493440d5b\") " pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:12.176699 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176648 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkkh2\" (UniqueName: \"kubernetes.io/projected/06952f51-6375-47cd-b827-88ef7b5d38ae-kube-api-access-mkkh2\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:12.176699 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.176699 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176695 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-socket-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.176891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-device-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.176891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176741 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.176891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176763 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d268c08-9ed4-4065-bb9a-59e46edb4820-host\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.176891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176786 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8d268c08-9ed4-4065-bb9a-59e46edb4820-serviceca\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.176891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsbnt\" (UniqueName: \"kubernetes.io/projected/8d268c08-9ed4-4065-bb9a-59e46edb4820-kube-api-access-dsbnt\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.176891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176830 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysconfig\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.176891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176850 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-host\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.176891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-registration-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-sys-fs\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176922 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-os-release\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-kubernetes\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176967 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-systemd\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.176988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-tmp\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177011 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e067270-9c93-492a-8247-37165df0c87d-hosts-file\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177053 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e067270-9c93-492a-8247-37165df0c87d-tmp-dir\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177093 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177122 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-run\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177169 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177197 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dzgr\" (UniqueName: \"kubernetes.io/projected/9e067270-9c93-492a-8247-37165df0c87d-kube-api-access-4dzgr\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177228 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnzdp\" (UniqueName: \"kubernetes.io/projected/fc8c28fa-c3a1-4e56-a531-e470dec2faca-kube-api-access-jnzdp\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177252 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50816dc4-c2f9-4346-b8fa-d8e493440d5b-agent-certs\") pod \"konnectivity-agent-j5x6b\" (UID: \"50816dc4-c2f9-4346-b8fa-d8e493440d5b\") " pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177275 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-modprobe-d\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177297 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysctl-conf\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cnibin\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177377 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177399 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysctl-d\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177436 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-tuned\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177473 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-var-lib-kubelet\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177497 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177522 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnx2w\" (UniqueName: \"kubernetes.io/projected/b9755c5e-ab28-49eb-9f63-02e7f64067ee-kube-api-access-gnx2w\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177546 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177603 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-sys\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-lib-modules\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177684 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvvb2\" (UniqueName: \"kubernetes.io/projected/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-kube-api-access-bvvb2\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177691 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177737 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bc2jb\"" Jun 03 16:21:12.177989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.177744 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 16:21:12.178570 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.178001 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 16:21:12.178570 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.178091 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 16:21:12.178570 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.178172 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 16:21:12.178570 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.178173 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 16:21:12.206707 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.206680 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:16:11 +0000 UTC" deadline="2028-02-05 18:28:48.231128642 +0000 UTC" Jun 03 16:21:12.206707 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.206701 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14690h7m36.024430548s" Jun 03 16:21:12.267163 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.267142 2576 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 16:21:12.278257 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278233 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-var-lib-kubelet\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.278380 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278264 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.278380 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-k8s-cni-cncf-io\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.278380 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278307 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.278380 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278365 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-var-lib-kubelet\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.278578 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278419 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.278578 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278445 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.278578 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278455 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-sys\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.278578 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278519 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-sys\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.278578 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278549 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-cni-bin\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278583 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-kubelet\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278610 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-ovnkube-script-lib\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278639 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278666 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkqdz\" (UniqueName: \"kubernetes.io/projected/54ce1f5d-d91c-4397-9d46-28ab18f21748-kube-api-access-qkqdz\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-log-socket\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-cni-bin\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278719 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278736 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-env-overrides\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278762 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8d268c08-9ed4-4065-bb9a-59e46edb4820-serviceca\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278789 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-registration-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.278812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278817 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-sys-fs\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278828 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278844 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-slash\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278883 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-os-release\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278911 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-kubernetes\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278919 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278934 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-sys-fs\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278942 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-systemd\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278972 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-tmp\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278989 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-systemd\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278936 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-registration-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.278998 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-kubernetes\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279002 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279029 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-run\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279047 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279074 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxpp6\" (UniqueName: \"kubernetes.io/projected/9a7e81d0-b84a-4301-a034-17a0005c76cb-kube-api-access-jxpp6\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-etc-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.279345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279138 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b150971-2aa3-4738-948a-256e18c8d0ef-ovn-node-metrics-cert\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279160 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4dzgr\" (UniqueName: \"kubernetes.io/projected/9e067270-9c93-492a-8247-37165df0c87d-kube-api-access-4dzgr\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279180 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jnzdp\" (UniqueName: \"kubernetes.io/projected/fc8c28fa-c3a1-4e56-a531-e470dec2faca-kube-api-access-jnzdp\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279207 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50816dc4-c2f9-4346-b8fa-d8e493440d5b-agent-certs\") pod \"konnectivity-agent-j5x6b\" (UID: \"50816dc4-c2f9-4346-b8fa-d8e493440d5b\") " pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279236 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-modprobe-d\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279238 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279292 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8d268c08-9ed4-4065-bb9a-59e46edb4820-serviceca\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279315 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-modprobe-d\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279369 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-os-release\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279412 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-run\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.279490 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.279581 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs podName:06952f51-6375-47cd-b827-88ef7b5d38ae nodeName:}" failed. No retries permitted until 2026-06-03 16:21:12.779536314 +0000 UTC m=+3.043600169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs") pod "network-metrics-daemon-89gzm" (UID: "06952f51-6375-47cd-b827-88ef7b5d38ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279599 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a7e81d0-b84a-4301-a034-17a0005c76cb-cni-binary-copy\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279655 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-systemd-units\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279693 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-node-log\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279725 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.280130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279750 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-tuned\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279773 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-netns\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279833 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279859 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gnx2w\" (UniqueName: \"kubernetes.io/projected/b9755c5e-ab28-49eb-9f63-02e7f64067ee-kube-api-access-gnx2w\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279889 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-cnibin\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/54ce1f5d-d91c-4397-9d46-28ab18f21748-iptables-alerter-script\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279935 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279956 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-cni-netd\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.279979 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-lib-modules\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280003 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bvvb2\" (UniqueName: \"kubernetes.io/projected/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-kube-api-access-bvvb2\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280026 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-socket-dir-parent\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280048 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-kubelet\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280086 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-var-lib-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280121 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-ovn\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280126 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280166 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-system-cni-dir\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.280852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280191 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50816dc4-c2f9-4346-b8fa-d8e493440d5b-konnectivity-ca\") pod \"konnectivity-agent-j5x6b\" (UID: \"50816dc4-c2f9-4346-b8fa-d8e493440d5b\") " pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280218 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mkkh2\" (UniqueName: \"kubernetes.io/projected/06952f51-6375-47cd-b827-88ef7b5d38ae-kube-api-access-mkkh2\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280242 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-socket-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280284 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-device-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280300 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-system-cni-dir\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280314 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-cni-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280342 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-lib-modules\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280354 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-cni-multus\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280399 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-device-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280427 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280451 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d268c08-9ed4-4065-bb9a-59e46edb4820-host\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280476 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dsbnt\" (UniqueName: \"kubernetes.io/projected/8d268c08-9ed4-4065-bb9a-59e46edb4820-kube-api-access-dsbnt\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280557 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9755c5e-ab28-49eb-9f63-02e7f64067ee-socket-dir\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280590 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280600 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d268c08-9ed4-4065-bb9a-59e46edb4820-host\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280626 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysconfig\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280655 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-host\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.281500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280683 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-daemon-config\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280692 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50816dc4-c2f9-4346-b8fa-d8e493440d5b-konnectivity-ca\") pod \"konnectivity-agent-j5x6b\" (UID: \"50816dc4-c2f9-4346-b8fa-d8e493440d5b\") " pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280727 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-etc-kubernetes\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280744 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysconfig\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-system-cni-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280778 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-os-release\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280790 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-host\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-run-netns\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280833 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-systemd\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280862 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e067270-9c93-492a-8247-37165df0c87d-hosts-file\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280887 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e067270-9c93-492a-8247-37165df0c87d-tmp-dir\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280910 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9e067270-9c93-492a-8247-37165df0c87d-hosts-file\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280915 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54ce1f5d-d91c-4397-9d46-28ab18f21748-host-slash\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280944 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysctl-conf\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280967 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-hostroot\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.280989 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-ovnkube-config\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281009 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psv5l\" (UniqueName: \"kubernetes.io/projected/2b150971-2aa3-4738-948a-256e18c8d0ef-kube-api-access-psv5l\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.282301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281031 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cnibin\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281049 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysctl-conf\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281055 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysctl-d\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281086 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-conf-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281103 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc8c28fa-c3a1-4e56-a531-e470dec2faca-cnibin\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281120 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-multus-certs\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281130 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e067270-9c93-492a-8247-37165df0c87d-tmp-dir\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.281213 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-sysctl-d\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.282711 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-tmp\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.283055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.282728 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-etc-tuned\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.283472 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.283109 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50816dc4-c2f9-4346-b8fa-d8e493440d5b-agent-certs\") pod \"konnectivity-agent-j5x6b\" (UID: \"50816dc4-c2f9-4346-b8fa-d8e493440d5b\") " pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:12.286879 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.286256 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:21:12.286879 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.286280 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:21:12.286879 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.286290 2576 projected.go:194] Error preparing data for projected volume kube-api-access-xw4xk for pod openshift-network-diagnostics/network-check-target-2jjqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:12.286879 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.286366 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk podName:b774cdc0-a68b-4b5a-9d9e-ed83bde9884f nodeName:}" failed. No retries permitted until 2026-06-03 16:21:12.78634933 +0000 UTC m=+3.050413204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xw4xk" (UniqueName: "kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk") pod "network-check-target-2jjqb" (UID: "b774cdc0-a68b-4b5a-9d9e-ed83bde9884f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:12.289196 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.289100 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnx2w\" (UniqueName: \"kubernetes.io/projected/b9755c5e-ab28-49eb-9f63-02e7f64067ee-kube-api-access-gnx2w\") pod \"aws-ebs-csi-driver-node-f8hcx\" (UID: \"b9755c5e-ab28-49eb-9f63-02e7f64067ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.289464 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.289409 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvvb2\" (UniqueName: \"kubernetes.io/projected/9f00b21d-53bf-4fe4-9518-8a5a5ccafe34-kube-api-access-bvvb2\") pod \"tuned-fs6bm\" (UID: \"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34\") " pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.289657 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.289614 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsbnt\" (UniqueName: \"kubernetes.io/projected/8d268c08-9ed4-4065-bb9a-59e46edb4820-kube-api-access-dsbnt\") pod \"node-ca-v7vzl\" (UID: \"8d268c08-9ed4-4065-bb9a-59e46edb4820\") " pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.289845 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.289827 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnzdp\" (UniqueName: \"kubernetes.io/projected/fc8c28fa-c3a1-4e56-a531-e470dec2faca-kube-api-access-jnzdp\") pod \"multus-additional-cni-plugins-jjcb8\" (UID: \"fc8c28fa-c3a1-4e56-a531-e470dec2faca\") " pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.289901 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.289860 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkkh2\" (UniqueName: \"kubernetes.io/projected/06952f51-6375-47cd-b827-88ef7b5d38ae-kube-api-access-mkkh2\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:12.290034 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.290014 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dzgr\" (UniqueName: \"kubernetes.io/projected/9e067270-9c93-492a-8247-37165df0c87d-kube-api-access-4dzgr\") pod \"node-resolver-pndzh\" (UID: \"9e067270-9c93-492a-8247-37165df0c87d\") " pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.303439 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.303415 2576 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:21:12.382191 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382154 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54ce1f5d-d91c-4397-9d46-28ab18f21748-host-slash\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.382191 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382194 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-hostroot\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382415 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382259 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54ce1f5d-d91c-4397-9d46-28ab18f21748-host-slash\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.382415 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382343 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-hostroot\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382415 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382392 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-ovnkube-config\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.382548 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382417 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-psv5l\" (UniqueName: \"kubernetes.io/projected/2b150971-2aa3-4738-948a-256e18c8d0ef-kube-api-access-psv5l\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.382548 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382434 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-conf-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382548 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382456 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-multus-certs\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382548 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382488 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-multus-certs\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382548 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382515 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-conf-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382548 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382513 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-k8s-cni-cncf-io\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382549 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-k8s-cni-cncf-io\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382568 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-cni-bin\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382593 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-kubelet\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382616 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-ovnkube-script-lib\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382667 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-kubelet\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382670 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-cni-bin\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382695 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qkqdz\" (UniqueName: \"kubernetes.io/projected/54ce1f5d-d91c-4397-9d46-28ab18f21748-kube-api-access-qkqdz\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382739 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-log-socket\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382763 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-cni-bin\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382806 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-cni-bin\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382805 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-log-socket\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.382836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382825 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-env-overrides\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382848 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-slash\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382891 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jxpp6\" (UniqueName: \"kubernetes.io/projected/9a7e81d0-b84a-4301-a034-17a0005c76cb-kube-api-access-jxpp6\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382920 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-slash\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.382976 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-etc-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383002 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383026 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b150971-2aa3-4738-948a-256e18c8d0ef-ovn-node-metrics-cert\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383048 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-etc-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383055 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a7e81d0-b84a-4301-a034-17a0005c76cb-cni-binary-copy\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383074 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383095 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-systemd-units\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383120 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-node-log\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383145 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-netns\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383166 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383156 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-systemd-units\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383186 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-cnibin\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383201 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/54ce1f5d-d91c-4397-9d46-28ab18f21748-iptables-alerter-script\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.383336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383216 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-run-netns\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383220 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383244 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-cni-netd\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383245 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-env-overrides\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383265 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-cnibin\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383268 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-socket-dir-parent\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383299 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383306 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383302 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-kubelet\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383216 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-node-log\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383351 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-kubelet\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383372 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-var-lib-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383379 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-cni-netd\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383399 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-ovn\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383417 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-var-lib-openvswitch\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383402 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-socket-dir-parent\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383440 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-cni-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383466 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-ovn\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383472 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-cni-multus\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383504 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-daemon-config\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383543 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-host-var-lib-cni-multus\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383576 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-etc-kubernetes\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383603 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-system-cni-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383640 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a7e81d0-b84a-4301-a034-17a0005c76cb-cni-binary-copy\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383644 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-etc-kubernetes\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383650 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-os-release\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383664 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-ovnkube-config\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383681 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-run-netns\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383696 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-cni-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383705 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-os-release\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383715 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-host-run-netns\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383720 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-systemd\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383742 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/54ce1f5d-d91c-4397-9d46-28ab18f21748-iptables-alerter-script\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383751 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a7e81d0-b84a-4301-a034-17a0005c76cb-system-cni-dir\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383789 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b150971-2aa3-4738-948a-256e18c8d0ef-run-systemd\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383829 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b150971-2aa3-4738-948a-256e18c8d0ef-ovnkube-script-lib\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.384981 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.383908 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a7e81d0-b84a-4301-a034-17a0005c76cb-multus-daemon-config\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.385765 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.385746 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b150971-2aa3-4738-948a-256e18c8d0ef-ovn-node-metrics-cert\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.390136 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.390116 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxpp6\" (UniqueName: \"kubernetes.io/projected/9a7e81d0-b84a-4301-a034-17a0005c76cb-kube-api-access-jxpp6\") pod \"multus-cmrwn\" (UID: \"9a7e81d0-b84a-4301-a034-17a0005c76cb\") " pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.390425 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.390397 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psv5l\" (UniqueName: \"kubernetes.io/projected/2b150971-2aa3-4738-948a-256e18c8d0ef-kube-api-access-psv5l\") pod \"ovnkube-node-68jfx\" (UID: \"2b150971-2aa3-4738-948a-256e18c8d0ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.390498 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.390435 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkqdz\" (UniqueName: \"kubernetes.io/projected/54ce1f5d-d91c-4397-9d46-28ab18f21748-kube-api-access-qkqdz\") pod \"iptables-alerter-cmrvv\" (UID: \"54ce1f5d-d91c-4397-9d46-28ab18f21748\") " pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.475361 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.475277 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" Jun 03 16:21:12.479922 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.479903 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pndzh" Jun 03 16:21:12.492614 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.492595 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" Jun 03 16:21:12.498132 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.498114 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" Jun 03 16:21:12.505680 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.505663 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v7vzl" Jun 03 16:21:12.511188 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.511167 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:12.516745 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.516727 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cmrwn" Jun 03 16:21:12.523175 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.523159 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cmrvv" Jun 03 16:21:12.528717 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.528694 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:12.787075 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.787045 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:12.787238 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:12.787085 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:12.787238 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.787195 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:21:12.787238 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.787222 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:21:12.787238 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.787232 2576 projected.go:194] Error preparing data for projected volume kube-api-access-xw4xk for pod openshift-network-diagnostics/network-check-target-2jjqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:12.787452 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.787275 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk podName:b774cdc0-a68b-4b5a-9d9e-ed83bde9884f nodeName:}" failed. No retries permitted until 2026-06-03 16:21:13.787260869 +0000 UTC m=+4.051324738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw4xk" (UniqueName: "kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk") pod "network-check-target-2jjqb" (UID: "b774cdc0-a68b-4b5a-9d9e-ed83bde9884f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:12.787452 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.787194 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:12.787452 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:12.787353 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs podName:06952f51-6375-47cd-b827-88ef7b5d38ae nodeName:}" failed. No retries permitted until 2026-06-03 16:21:13.787336118 +0000 UTC m=+4.051399978 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs") pod "network-metrics-daemon-89gzm" (UID: "06952f51-6375-47cd-b827-88ef7b5d38ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:12.899274 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:12.899242 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc8c28fa_c3a1_4e56_a531_e470dec2faca.slice/crio-af1844a1144816ac82ac4438be46907bbeed9378b937daf885162ad2d211508d WatchSource:0}: Error finding container af1844a1144816ac82ac4438be46907bbeed9378b937daf885162ad2d211508d: Status 404 returned error can't find the container with id af1844a1144816ac82ac4438be46907bbeed9378b937daf885162ad2d211508d Jun 03 16:21:12.901146 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:12.901122 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e067270_9c93_492a_8247_37165df0c87d.slice/crio-b43a05903efe1da378558376af05d420cd149e0183ee9774b6869029f91a76e8 WatchSource:0}: Error finding container b43a05903efe1da378558376af05d420cd149e0183ee9774b6869029f91a76e8: Status 404 returned error can't find the container with id b43a05903efe1da378558376af05d420cd149e0183ee9774b6869029f91a76e8 Jun 03 16:21:12.903691 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:12.903669 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f00b21d_53bf_4fe4_9518_8a5a5ccafe34.slice/crio-45cd028981b240d2fe56223cbb3bc62e2d5e088ed2cbad7686ec8fec979f3eaf WatchSource:0}: Error finding container 45cd028981b240d2fe56223cbb3bc62e2d5e088ed2cbad7686ec8fec979f3eaf: Status 404 returned error can't find the container with id 45cd028981b240d2fe56223cbb3bc62e2d5e088ed2cbad7686ec8fec979f3eaf Jun 03 16:21:12.904789 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:12.904732 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d268c08_9ed4_4065_bb9a_59e46edb4820.slice/crio-67c4041aa39ba99c8ff5a0a6d472014e4614d2c66aba4db75ba8044bd2f1463a WatchSource:0}: Error finding container 67c4041aa39ba99c8ff5a0a6d472014e4614d2c66aba4db75ba8044bd2f1463a: Status 404 returned error can't find the container with id 67c4041aa39ba99c8ff5a0a6d472014e4614d2c66aba4db75ba8044bd2f1463a Jun 03 16:21:12.905543 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:12.905522 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a7e81d0_b84a_4301_a034_17a0005c76cb.slice/crio-74227a52962eb3d0f6abcc1dd6f56f948df1df4c2188255803cb78beef9e1ef5 WatchSource:0}: Error finding container 74227a52962eb3d0f6abcc1dd6f56f948df1df4c2188255803cb78beef9e1ef5: Status 404 returned error can't find the container with id 74227a52962eb3d0f6abcc1dd6f56f948df1df4c2188255803cb78beef9e1ef5 Jun 03 16:21:12.906305 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:12.906287 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b150971_2aa3_4738_948a_256e18c8d0ef.slice/crio-26f46070fd2953772ef24f10cc97117e0b32c12df6afc15d91aec0f14db8dfe5 WatchSource:0}: Error finding container 26f46070fd2953772ef24f10cc97117e0b32c12df6afc15d91aec0f14db8dfe5: Status 404 returned error can't find the container with id 26f46070fd2953772ef24f10cc97117e0b32c12df6afc15d91aec0f14db8dfe5 Jun 03 16:21:12.907521 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:12.907423 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50816dc4_c2f9_4346_b8fa_d8e493440d5b.slice/crio-f45f1895094d821f7dde051ca292a1bccfe348f98e0755b6a7ccbee5941b8db1 WatchSource:0}: Error finding container f45f1895094d821f7dde051ca292a1bccfe348f98e0755b6a7ccbee5941b8db1: Status 404 returned error can't find the container with id f45f1895094d821f7dde051ca292a1bccfe348f98e0755b6a7ccbee5941b8db1 Jun 03 16:21:13.207238 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.207080 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:16:11 +0000 UTC" deadline="2028-02-07 19:57:49.583730508 +0000 UTC" Jun 03 16:21:13.207581 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.207240 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14739h36m36.376493945s" Jun 03 16:21:13.309069 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.309036 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:13.309271 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:13.309172 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:13.321366 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.321300 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v7vzl" event={"ID":"8d268c08-9ed4-4065-bb9a-59e46edb4820","Type":"ContainerStarted","Data":"67c4041aa39ba99c8ff5a0a6d472014e4614d2c66aba4db75ba8044bd2f1463a"} Jun 03 16:21:13.322926 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.322899 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" event={"ID":"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34","Type":"ContainerStarted","Data":"45cd028981b240d2fe56223cbb3bc62e2d5e088ed2cbad7686ec8fec979f3eaf"} Jun 03 16:21:13.324617 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.324593 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerStarted","Data":"af1844a1144816ac82ac4438be46907bbeed9378b937daf885162ad2d211508d"} Jun 03 16:21:13.327381 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.327363 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" event={"ID":"167b61c3f33681bb1d2fd6165981fb2f","Type":"ContainerStarted","Data":"b6261b83c9ad5e828fcd1b1e4cdb4aa3551913aa49d72fdf4bc9fa8e35808525"} Jun 03 16:21:13.329370 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.328848 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" event={"ID":"b9755c5e-ab28-49eb-9f63-02e7f64067ee","Type":"ContainerStarted","Data":"c58e2f96345b6f97c11108cf3d71abf893291dc1350fdd1cf4652d1b7dd04059"} Jun 03 16:21:13.331166 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.329857 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cmrvv" event={"ID":"54ce1f5d-d91c-4397-9d46-28ab18f21748","Type":"ContainerStarted","Data":"d4e7dab650c6c1c1acfec65010b0bb51cf903df809d7d86c7cfb8df7b627c623"} Jun 03 16:21:13.332846 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.332816 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j5x6b" event={"ID":"50816dc4-c2f9-4346-b8fa-d8e493440d5b","Type":"ContainerStarted","Data":"f45f1895094d821f7dde051ca292a1bccfe348f98e0755b6a7ccbee5941b8db1"} Jun 03 16:21:13.336150 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.336097 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"26f46070fd2953772ef24f10cc97117e0b32c12df6afc15d91aec0f14db8dfe5"} Jun 03 16:21:13.338000 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.337487 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cmrwn" event={"ID":"9a7e81d0-b84a-4301-a034-17a0005c76cb","Type":"ContainerStarted","Data":"74227a52962eb3d0f6abcc1dd6f56f948df1df4c2188255803cb78beef9e1ef5"} Jun 03 16:21:13.341528 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.341489 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pndzh" event={"ID":"9e067270-9c93-492a-8247-37165df0c87d","Type":"ContainerStarted","Data":"b43a05903efe1da378558376af05d420cd149e0183ee9774b6869029f91a76e8"} Jun 03 16:21:13.343413 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.343349 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-141.ec2.internal" podStartSLOduration=2.343335775 podStartE2EDuration="2.343335775s" podCreationTimestamp="2026-06-03 16:21:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:21:13.342556938 +0000 UTC m=+3.606620815" watchObservedRunningTime="2026-06-03 16:21:13.343335775 +0000 UTC m=+3.607399648" Jun 03 16:21:13.345846 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.345823 2576 generic.go:358] "Generic (PLEG): container finished" podID="b690a59b3980ddc7968d917c387fd722" containerID="b25b2a7ddf1d50ed07c7aa090a2a8c89152b6fef1f58809a5ef86447ef248162" exitCode=0 Jun 03 16:21:13.345920 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.345857 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" event={"ID":"b690a59b3980ddc7968d917c387fd722","Type":"ContainerDied","Data":"b25b2a7ddf1d50ed07c7aa090a2a8c89152b6fef1f58809a5ef86447ef248162"} Jun 03 16:21:13.807233 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.807167 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:13.807233 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:13.807234 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:13.807505 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:13.807405 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:13.807505 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:13.807475 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs podName:06952f51-6375-47cd-b827-88ef7b5d38ae nodeName:}" failed. No retries permitted until 2026-06-03 16:21:15.807454694 +0000 UTC m=+6.071518567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs") pod "network-metrics-daemon-89gzm" (UID: "06952f51-6375-47cd-b827-88ef7b5d38ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:13.807911 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:13.807893 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:21:13.807994 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:13.807915 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:21:13.807994 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:13.807928 2576 projected.go:194] Error preparing data for projected volume kube-api-access-xw4xk for pod openshift-network-diagnostics/network-check-target-2jjqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:13.807994 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:13.807970 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk podName:b774cdc0-a68b-4b5a-9d9e-ed83bde9884f nodeName:}" failed. No retries permitted until 2026-06-03 16:21:15.807956183 +0000 UTC m=+6.072020042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw4xk" (UniqueName: "kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk") pod "network-check-target-2jjqb" (UID: "b774cdc0-a68b-4b5a-9d9e-ed83bde9884f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:14.309557 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:14.309494 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:14.310208 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:14.309628 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:14.367638 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:14.367487 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" event={"ID":"b690a59b3980ddc7968d917c387fd722","Type":"ContainerStarted","Data":"c8a8a328fff4b9420eb16a1925f79e4b88125e53c87d3f494a0c13b2ad483f6c"} Jun 03 16:21:15.309864 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:15.309378 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:15.309864 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:15.309526 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:15.824347 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:15.824247 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:15.824347 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:15.824295 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:15.824587 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:15.824406 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:15.824587 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:15.824417 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:21:15.824587 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:15.824432 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:21:15.824587 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:15.824445 2576 projected.go:194] Error preparing data for projected volume kube-api-access-xw4xk for pod openshift-network-diagnostics/network-check-target-2jjqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:15.824587 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:15.824471 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs podName:06952f51-6375-47cd-b827-88ef7b5d38ae nodeName:}" failed. No retries permitted until 2026-06-03 16:21:19.824452116 +0000 UTC m=+10.088515988 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs") pod "network-metrics-daemon-89gzm" (UID: "06952f51-6375-47cd-b827-88ef7b5d38ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:15.824587 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:15.824488 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk podName:b774cdc0-a68b-4b5a-9d9e-ed83bde9884f nodeName:}" failed. No retries permitted until 2026-06-03 16:21:19.824479503 +0000 UTC m=+10.088543356 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw4xk" (UniqueName: "kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk") pod "network-check-target-2jjqb" (UID: "b774cdc0-a68b-4b5a-9d9e-ed83bde9884f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:16.309496 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:16.309464 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:16.309676 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:16.309581 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:17.309488 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:17.309452 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:17.310038 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:17.309600 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:18.308465 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:18.308429 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:18.308630 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:18.308559 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:19.309255 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:19.309226 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:19.309777 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:19.309368 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:19.853779 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:19.853741 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:19.853965 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:19.853794 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:19.853965 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:19.853911 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:19.853965 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:19.853936 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:21:19.853965 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:19.853963 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:21:19.853965 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:19.853971 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs podName:06952f51-6375-47cd-b827-88ef7b5d38ae nodeName:}" failed. No retries permitted until 2026-06-03 16:21:27.853952054 +0000 UTC m=+18.118015913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs") pod "network-metrics-daemon-89gzm" (UID: "06952f51-6375-47cd-b827-88ef7b5d38ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:19.854253 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:19.853978 2576 projected.go:194] Error preparing data for projected volume kube-api-access-xw4xk for pod openshift-network-diagnostics/network-check-target-2jjqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:19.854253 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:19.854029 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk podName:b774cdc0-a68b-4b5a-9d9e-ed83bde9884f nodeName:}" failed. No retries permitted until 2026-06-03 16:21:27.854013626 +0000 UTC m=+18.118077485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw4xk" (UniqueName: "kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk") pod "network-check-target-2jjqb" (UID: "b774cdc0-a68b-4b5a-9d9e-ed83bde9884f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:20.312550 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:20.312517 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:20.312983 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:20.312640 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:21.309172 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:21.309140 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:21.309348 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:21.309266 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:22.312095 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:22.311971 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:22.312555 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:22.312109 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:23.309497 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.309397 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:23.309661 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:23.309530 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:23.391180 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.391140 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" event={"ID":"b9755c5e-ab28-49eb-9f63-02e7f64067ee","Type":"ContainerStarted","Data":"0112b2cd20a4f42b9362a958e4fc8c310824b08e489429120e6ea829c63a8557"} Jun 03 16:21:23.392563 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.392535 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j5x6b" event={"ID":"50816dc4-c2f9-4346-b8fa-d8e493440d5b","Type":"ContainerStarted","Data":"ae419b1bb932dba41eb16db4fc2bd6d4dca50cb768cdf94253724123fa52ef12"} Jun 03 16:21:23.393972 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.393950 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pndzh" event={"ID":"9e067270-9c93-492a-8247-37165df0c87d","Type":"ContainerStarted","Data":"d3533dccc14dc0b43ee9e6bfb42ab828031235b6389932037e6fc598cd3c72a6"} Jun 03 16:21:23.395246 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.395220 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v7vzl" event={"ID":"8d268c08-9ed4-4065-bb9a-59e46edb4820","Type":"ContainerStarted","Data":"6350aa345eb600c8403a5bc7a7d59ed36b3b36b1a027885bf098df7104548000"} Jun 03 16:21:23.396451 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.396429 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" event={"ID":"9f00b21d-53bf-4fe4-9518-8a5a5ccafe34","Type":"ContainerStarted","Data":"092d3783413529f8b5fa1123f38c9b303c0330cffb8bd4707f0a0408bdaf31cb"} Jun 03 16:21:23.397710 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.397690 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerStarted","Data":"77576785381f9f0c7db1014c24454f000063d3365fa2832683223f86222731fd"} Jun 03 16:21:23.409919 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.409868 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-141.ec2.internal" podStartSLOduration=12.409856124 podStartE2EDuration="12.409856124s" podCreationTimestamp="2026-06-03 16:21:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:21:14.384280506 +0000 UTC m=+4.648344381" watchObservedRunningTime="2026-06-03 16:21:23.409856124 +0000 UTC m=+13.673919999" Jun 03 16:21:23.426492 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.426460 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j5x6b" podStartSLOduration=3.909923407 podStartE2EDuration="13.426450103s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.911116686 +0000 UTC m=+3.175180540" lastFinishedPulling="2026-06-03 16:21:22.427643379 +0000 UTC m=+12.691707236" observedRunningTime="2026-06-03 16:21:23.409775842 +0000 UTC m=+13.673839716" watchObservedRunningTime="2026-06-03 16:21:23.426450103 +0000 UTC m=+13.690514000" Jun 03 16:21:23.448396 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.448360 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fs6bm" podStartSLOduration=3.90031341 podStartE2EDuration="13.448347227s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.905075608 +0000 UTC m=+3.169139478" lastFinishedPulling="2026-06-03 16:21:22.453109427 +0000 UTC m=+12.717173295" observedRunningTime="2026-06-03 16:21:23.426571355 +0000 UTC m=+13.690635225" watchObservedRunningTime="2026-06-03 16:21:23.448347227 +0000 UTC m=+13.712411118" Jun 03 16:21:23.485345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.485180 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v7vzl" podStartSLOduration=3.9493990329999997 podStartE2EDuration="13.485171089s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.906261178 +0000 UTC m=+3.170325045" lastFinishedPulling="2026-06-03 16:21:22.44203303 +0000 UTC m=+12.706097101" observedRunningTime="2026-06-03 16:21:23.46947499 +0000 UTC m=+13.733538878" watchObservedRunningTime="2026-06-03 16:21:23.485171089 +0000 UTC m=+13.749234964" Jun 03 16:21:23.485751 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:23.485722 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-pndzh" podStartSLOduration=3.95780215 podStartE2EDuration="13.485713942s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.902845048 +0000 UTC m=+3.166908901" lastFinishedPulling="2026-06-03 16:21:22.430756825 +0000 UTC m=+12.694820693" observedRunningTime="2026-06-03 16:21:23.484980242 +0000 UTC m=+13.749044111" watchObservedRunningTime="2026-06-03 16:21:23.485713942 +0000 UTC m=+13.749777817" Jun 03 16:21:24.309300 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:24.309267 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:24.309494 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:24.309418 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:24.401006 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:24.400966 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cmrvv" event={"ID":"54ce1f5d-d91c-4397-9d46-28ab18f21748","Type":"ContainerStarted","Data":"24774bf54a9607d9f03e0df47c9acde86260ba27c8c775caedc92fca43f91f21"} Jun 03 16:21:24.418264 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:24.418220 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cmrvv" podStartSLOduration=4.875358907 podStartE2EDuration="14.418208462s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.913419539 +0000 UTC m=+3.177483403" lastFinishedPulling="2026-06-03 16:21:22.456269105 +0000 UTC m=+12.720332958" observedRunningTime="2026-06-03 16:21:24.417786562 +0000 UTC m=+14.681850629" watchObservedRunningTime="2026-06-03 16:21:24.418208462 +0000 UTC m=+14.682272336" Jun 03 16:21:25.203051 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:25.203017 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:25.203708 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:25.203688 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:25.308901 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:25.308871 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:25.309068 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:25.308978 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:25.345476 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:25.345449 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:25.346051 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:25.346027 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j5x6b" Jun 03 16:21:26.308579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:26.308546 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:26.309244 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:26.308678 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:27.308916 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:27.308890 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:27.309624 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:27.309018 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:27.407828 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:27.407793 2576 generic.go:358] "Generic (PLEG): container finished" podID="fc8c28fa-c3a1-4e56-a531-e470dec2faca" containerID="77576785381f9f0c7db1014c24454f000063d3365fa2832683223f86222731fd" exitCode=0 Jun 03 16:21:27.407965 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:27.407834 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerDied","Data":"77576785381f9f0c7db1014c24454f000063d3365fa2832683223f86222731fd"} Jun 03 16:21:27.917241 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:27.917208 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:27.917241 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:27.917247 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:27.917474 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:27.917387 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:27.917474 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:27.917393 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:21:27.917474 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:27.917413 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:21:27.917474 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:27.917426 2576 projected.go:194] Error preparing data for projected volume kube-api-access-xw4xk for pod openshift-network-diagnostics/network-check-target-2jjqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:27.917474 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:27.917446 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs podName:06952f51-6375-47cd-b827-88ef7b5d38ae nodeName:}" failed. No retries permitted until 2026-06-03 16:21:43.917432308 +0000 UTC m=+34.181496161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs") pod "network-metrics-daemon-89gzm" (UID: "06952f51-6375-47cd-b827-88ef7b5d38ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:27.917647 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:27.917484 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk podName:b774cdc0-a68b-4b5a-9d9e-ed83bde9884f nodeName:}" failed. No retries permitted until 2026-06-03 16:21:43.917471229 +0000 UTC m=+34.181535082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw4xk" (UniqueName: "kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk") pod "network-check-target-2jjqb" (UID: "b774cdc0-a68b-4b5a-9d9e-ed83bde9884f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:28.309027 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:28.308993 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:28.309573 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:28.309112 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:29.309405 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:29.309372 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:29.309814 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:29.309508 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:30.309977 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:30.309950 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:30.310344 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:30.310021 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:30.794343 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:30.794094 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 16:21:31.232159 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.231831 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T16:21:30.794271852Z","UUID":"e6aa78a0-e519-4308-bf71-f7cfd4879d3a","Handler":null,"Name":"","Endpoint":""} Jun 03 16:21:31.234080 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.234051 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 16:21:31.234080 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.234084 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 16:21:31.308690 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.308655 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:31.308818 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:31.308784 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:31.416455 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.416421 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" event={"ID":"b9755c5e-ab28-49eb-9f63-02e7f64067ee","Type":"ContainerStarted","Data":"d441331f785e5ece1deadf41f5f5ae48fc393b525a7d2487c218a62343fa3fd1"} Jun 03 16:21:31.419583 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.419552 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"323ee0eac05ecf22273d268320fa54493f18ad333c3b709691b0c4786e7d82c0"} Jun 03 16:21:31.419690 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.419590 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"0190dae72c892f0d0bb7598d154a01d4f8c9fd00fb8793a63945e08f884e33bc"} Jun 03 16:21:31.419690 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.419605 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"509f14d75902ca62071fbd731e6911edf703536095e99e64f15de8ec4fab0ad7"} Jun 03 16:21:31.419690 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.419622 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"a83bb2551d6c84da4bbf88422dbe6cefe2b39b5225b78d1e3995bd1348c2202b"} Jun 03 16:21:31.419690 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.419638 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"a227a70122239dcd9958fd383d25ed4bb8a6970e116736318a0dff57fb15c297"} Jun 03 16:21:31.419690 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.419652 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"28f5b9d019611c722e85977a37f7acae4c106600b99aa4c1b51137673a18e80c"} Jun 03 16:21:31.421088 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:31.421063 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cmrwn" event={"ID":"9a7e81d0-b84a-4301-a034-17a0005c76cb","Type":"ContainerStarted","Data":"1e010eb905d74723a63b4cb365d49242ca27e43bf3733f3b6ebc2c716293a325"} Jun 03 16:21:32.308827 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:32.308801 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:32.309117 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:32.308909 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:32.424822 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:32.424790 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" event={"ID":"b9755c5e-ab28-49eb-9f63-02e7f64067ee","Type":"ContainerStarted","Data":"215bc8e676acac723ff5c754fca0197ceef7dab42584991327a02717479caec7"} Jun 03 16:21:32.446771 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:32.446726 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8hcx" podStartSLOduration=3.732692106 podStartE2EDuration="22.446712639s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.913513977 +0000 UTC m=+3.177577831" lastFinishedPulling="2026-06-03 16:21:31.627534508 +0000 UTC m=+21.891598364" observedRunningTime="2026-06-03 16:21:32.44665444 +0000 UTC m=+22.710718325" watchObservedRunningTime="2026-06-03 16:21:32.446712639 +0000 UTC m=+22.710776513" Jun 03 16:21:32.447117 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:32.447096 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-cmrwn" podStartSLOduration=4.880888715 podStartE2EDuration="22.447091747s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.907644692 +0000 UTC m=+3.171708552" lastFinishedPulling="2026-06-03 16:21:30.473847731 +0000 UTC m=+20.737911584" observedRunningTime="2026-06-03 16:21:31.439083117 +0000 UTC m=+21.703146992" watchObservedRunningTime="2026-06-03 16:21:32.447091747 +0000 UTC m=+22.711155621" Jun 03 16:21:33.309356 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:33.309314 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:33.309490 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:33.309436 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:33.429177 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:33.429096 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"3351341f36a124a55bac88e97f60ad2a4072dcdb384d8bbbf66e8c7d594e348a"} Jun 03 16:21:34.309042 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:34.309015 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:34.309223 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:34.309137 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:35.309381 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:35.309353 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:35.309800 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:35.309450 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:35.513190 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:35.513173 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pndzh_9e067270-9c93-492a-8247-37165df0c87d/dns-node-resolver/0.log" Jun 03 16:21:36.309062 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:36.309031 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:36.309216 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:36.309146 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:36.437645 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:36.437617 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" event={"ID":"2b150971-2aa3-4738-948a-256e18c8d0ef","Type":"ContainerStarted","Data":"5ff0b13f18b5a28778a616273fc9aee45adadd60c004f12c4aef70e0a571166f"} Jun 03 16:21:36.438404 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:36.437877 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:36.439212 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:36.439192 2576 generic.go:358] "Generic (PLEG): container finished" podID="fc8c28fa-c3a1-4e56-a531-e470dec2faca" containerID="26fb5fd66aee61bf5b687f62626169b59dd29adffde2304c7ccfd27541eaef9b" exitCode=0 Jun 03 16:21:36.439283 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:36.439220 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerDied","Data":"26fb5fd66aee61bf5b687f62626169b59dd29adffde2304c7ccfd27541eaef9b"} Jun 03 16:21:36.452759 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:36.452741 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:36.465397 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:36.465362 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" podStartSLOduration=8.903712494 podStartE2EDuration="26.465352699s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.908061694 +0000 UTC m=+3.172125553" lastFinishedPulling="2026-06-03 16:21:30.469701904 +0000 UTC m=+20.733765758" observedRunningTime="2026-06-03 16:21:36.465102617 +0000 UTC m=+26.729166491" watchObservedRunningTime="2026-06-03 16:21:36.465352699 +0000 UTC m=+26.729416573" Jun 03 16:21:36.700732 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:36.700557 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v7vzl_8d268c08-9ed4-4065-bb9a-59e46edb4820/node-ca/0.log" Jun 03 16:21:37.308595 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:37.308566 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:37.308788 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:37.308696 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:37.441937 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:37.441913 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:37.442269 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:37.441947 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:37.457431 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:37.457381 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:21:37.704928 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:37.704859 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-89gzm"] Jun 03 16:21:37.705084 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:37.704952 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:37.705084 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:37.705040 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:37.709087 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:37.709051 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2jjqb"] Jun 03 16:21:37.709200 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:37.709155 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:37.709252 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:37.709238 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:38.445157 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:38.445118 2576 generic.go:358] "Generic (PLEG): container finished" podID="fc8c28fa-c3a1-4e56-a531-e470dec2faca" containerID="e2f87f610acb6dad55a36bebf6e920b856ad70e289be8360694c06b6f2e8d741" exitCode=0 Jun 03 16:21:38.445587 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:38.445195 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerDied","Data":"e2f87f610acb6dad55a36bebf6e920b856ad70e289be8360694c06b6f2e8d741"} Jun 03 16:21:39.309194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:39.309164 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:39.309316 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:39.309225 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:39.309389 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:39.309332 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:39.309435 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:39.309421 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:40.450232 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:40.450205 2576 generic.go:358] "Generic (PLEG): container finished" podID="fc8c28fa-c3a1-4e56-a531-e470dec2faca" containerID="f6ab9fe31993c9a695ce7628b67fc4661ddb0f2fadc76ee35256adfc16db2b80" exitCode=0 Jun 03 16:21:40.450600 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:40.450243 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerDied","Data":"f6ab9fe31993c9a695ce7628b67fc4661ddb0f2fadc76ee35256adfc16db2b80"} Jun 03 16:21:41.309369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:41.309335 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:41.309521 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:41.309342 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:41.309521 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:41.309460 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:41.309647 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:41.309546 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:43.308802 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:43.308769 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:43.308802 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:43.308800 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:43.309308 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:43.308878 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:43.309308 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:43.309018 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:43.936767 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:43.936729 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:43.936767 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:43.936770 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:43.937035 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:43.936893 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:43.937035 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:43.936900 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:21:43.937035 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:43.936923 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:21:43.937035 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:43.936937 2576 projected.go:194] Error preparing data for projected volume kube-api-access-xw4xk for pod openshift-network-diagnostics/network-check-target-2jjqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:43.937035 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:43.936954 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs podName:06952f51-6375-47cd-b827-88ef7b5d38ae nodeName:}" failed. No retries permitted until 2026-06-03 16:22:15.936934523 +0000 UTC m=+66.200998379 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs") pod "network-metrics-daemon-89gzm" (UID: "06952f51-6375-47cd-b827-88ef7b5d38ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:21:43.937035 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:43.936984 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk podName:b774cdc0-a68b-4b5a-9d9e-ed83bde9884f nodeName:}" failed. No retries permitted until 2026-06-03 16:22:15.936968396 +0000 UTC m=+66.201032251 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw4xk" (UniqueName: "kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk") pod "network-check-target-2jjqb" (UID: "b774cdc0-a68b-4b5a-9d9e-ed83bde9884f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:21:45.309092 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:45.309062 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:45.309649 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:45.309066 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:45.309649 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:45.309178 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:45.309649 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:45.309251 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:46.461893 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:46.461813 2576 generic.go:358] "Generic (PLEG): container finished" podID="fc8c28fa-c3a1-4e56-a531-e470dec2faca" containerID="b412dd64ab9b6f7beaeac319e457c23af790e65ba7dbb8d6b58d5f4d148430a0" exitCode=0 Jun 03 16:21:46.461893 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:46.461858 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerDied","Data":"b412dd64ab9b6f7beaeac319e457c23af790e65ba7dbb8d6b58d5f4d148430a0"} Jun 03 16:21:47.309233 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:47.309169 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:47.309380 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:47.309255 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:47.309380 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:47.309173 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:47.309380 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:47.309348 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:47.465866 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:47.465839 2576 generic.go:358] "Generic (PLEG): container finished" podID="fc8c28fa-c3a1-4e56-a531-e470dec2faca" containerID="44ab296cba3c9f861b3198dec5950853ffa7d2c3438dd2d5320b515df700f0bd" exitCode=0 Jun 03 16:21:47.466178 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:47.465871 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerDied","Data":"44ab296cba3c9f861b3198dec5950853ffa7d2c3438dd2d5320b515df700f0bd"} Jun 03 16:21:48.470335 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:48.470295 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" event={"ID":"fc8c28fa-c3a1-4e56-a531-e470dec2faca","Type":"ContainerStarted","Data":"779cc1d3dda8b7353778298f4a0bc368baf36e82423319862fe61a099d1c1c04"} Jun 03 16:21:48.499618 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:48.499526 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jjcb8" podStartSLOduration=5.295698412 podStartE2EDuration="38.499511879s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:21:12.90183074 +0000 UTC m=+3.165894611" lastFinishedPulling="2026-06-03 16:21:46.105644223 +0000 UTC m=+36.369708078" observedRunningTime="2026-06-03 16:21:48.498135216 +0000 UTC m=+38.762199091" watchObservedRunningTime="2026-06-03 16:21:48.499511879 +0000 UTC m=+38.763575754" Jun 03 16:21:49.309129 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:49.309101 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:49.309295 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:49.309183 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:49.309295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:49.309270 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:49.309443 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:49.309393 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:51.308589 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:51.308553 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:51.309022 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:51.308553 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:51.309022 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:51.308660 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:51.309022 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:51.308719 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:53.308833 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:53.308793 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:53.309217 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:53.308805 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:53.309217 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:53.308893 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:53.309217 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:53.308991 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:55.308409 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:55.308380 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:55.308876 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:55.308380 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:55.308876 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:55.308482 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:55.308876 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:55.308562 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:57.308617 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.308554 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:57.308939 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.308554 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:57.308939 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:57.308642 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2jjqb" podUID="b774cdc0-a68b-4b5a-9d9e-ed83bde9884f" Jun 03 16:21:57.308939 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:21:57.308745 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89gzm" podUID="06952f51-6375-47cd-b827-88ef7b5d38ae" Jun 03 16:21:57.594015 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.593942 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-141.ec2.internal" event="NodeReady" Jun 03 16:21:57.594157 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.594053 2576 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 16:21:57.641395 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.641367 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-59bdh"] Jun 03 16:21:57.647559 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.647542 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-scd2b"] Jun 03 16:21:57.647694 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.647680 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.650014 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.649994 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 16:21:57.650116 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.650099 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hc7pk\"" Jun 03 16:21:57.650405 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.650392 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 16:21:57.650584 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.650570 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.652853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.652822 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 16:21:57.652961 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.652859 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 16:21:57.653358 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.653079 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tnvqm\"" Jun 03 16:21:57.653358 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.653150 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 16:21:57.653358 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.653180 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 16:21:57.653532 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.653434 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-59bdh"] Jun 03 16:21:57.662244 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.662216 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-scd2b"] Jun 03 16:21:57.665017 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.664997 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m458w"] Jun 03 16:21:57.668156 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.668136 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m458w" Jun 03 16:21:57.671503 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.671486 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 16:21:57.671636 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.671619 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ntnfn\"" Jun 03 16:21:57.671715 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.671700 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 16:21:57.671775 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.671719 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 16:21:57.676521 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.676503 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m458w"] Jun 03 16:21:57.739499 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739473 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-data-volume\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.739613 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739502 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-tmp-dir\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.739613 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739542 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.739613 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739568 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.739613 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739607 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-crio-socket\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.739734 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739669 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shdj2\" (UniqueName: \"kubernetes.io/projected/35619cd5-0a7d-4752-bbc3-321dd5e1b1da-kube-api-access-shdj2\") pod \"ingress-canary-m458w\" (UID: \"35619cd5-0a7d-4752-bbc3-321dd5e1b1da\") " pod="openshift-ingress-canary/ingress-canary-m458w" Jun 03 16:21:57.739734 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlv8f\" (UniqueName: \"kubernetes.io/projected/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-kube-api-access-rlv8f\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.739734 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-config-volume\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.739817 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wffqg\" (UniqueName: \"kubernetes.io/projected/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-kube-api-access-wffqg\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.739817 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739754 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35619cd5-0a7d-4752-bbc3-321dd5e1b1da-cert\") pod \"ingress-canary-m458w\" (UID: \"35619cd5-0a7d-4752-bbc3-321dd5e1b1da\") " pod="openshift-ingress-canary/ingress-canary-m458w" Jun 03 16:21:57.739817 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.739769 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-metrics-tls\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.840775 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840753 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-data-volume\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.840849 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840780 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-tmp-dir\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.840849 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840806 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.840849 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840824 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.840978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840859 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-crio-socket\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.840978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840884 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-shdj2\" (UniqueName: \"kubernetes.io/projected/35619cd5-0a7d-4752-bbc3-321dd5e1b1da-kube-api-access-shdj2\") pod \"ingress-canary-m458w\" (UID: \"35619cd5-0a7d-4752-bbc3-321dd5e1b1da\") " pod="openshift-ingress-canary/ingress-canary-m458w" Jun 03 16:21:57.840978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840909 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rlv8f\" (UniqueName: \"kubernetes.io/projected/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-kube-api-access-rlv8f\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.840978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840930 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-config-volume\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.840978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.840953 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wffqg\" (UniqueName: \"kubernetes.io/projected/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-kube-api-access-wffqg\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.841151 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.841068 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-crio-socket\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.841151 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.841096 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-tmp-dir\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.841151 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.841108 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-data-volume\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.841151 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.841127 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35619cd5-0a7d-4752-bbc3-321dd5e1b1da-cert\") pod \"ingress-canary-m458w\" (UID: \"35619cd5-0a7d-4752-bbc3-321dd5e1b1da\") " pod="openshift-ingress-canary/ingress-canary-m458w" Jun 03 16:21:57.841339 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.841159 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-metrics-tls\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.841588 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.841566 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-config-volume\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.841588 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.841587 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.844817 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.844771 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-metrics-tls\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.844883 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.844823 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.844883 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.844835 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35619cd5-0a7d-4752-bbc3-321dd5e1b1da-cert\") pod \"ingress-canary-m458w\" (UID: \"35619cd5-0a7d-4752-bbc3-321dd5e1b1da\") " pod="openshift-ingress-canary/ingress-canary-m458w" Jun 03 16:21:57.848880 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.848860 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shdj2\" (UniqueName: \"kubernetes.io/projected/35619cd5-0a7d-4752-bbc3-321dd5e1b1da-kube-api-access-shdj2\") pod \"ingress-canary-m458w\" (UID: \"35619cd5-0a7d-4752-bbc3-321dd5e1b1da\") " pod="openshift-ingress-canary/ingress-canary-m458w" Jun 03 16:21:57.850038 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.850019 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wffqg\" (UniqueName: \"kubernetes.io/projected/e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d-kube-api-access-wffqg\") pod \"dns-default-59bdh\" (UID: \"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d\") " pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.850127 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.850104 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlv8f\" (UniqueName: \"kubernetes.io/projected/4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8-kube-api-access-rlv8f\") pod \"insights-runtime-extractor-scd2b\" (UID: \"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8\") " pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.957737 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.957709 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-59bdh" Jun 03 16:21:57.963422 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.963398 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-scd2b" Jun 03 16:21:57.975661 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:57.975637 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m458w" Jun 03 16:21:58.113967 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:58.113898 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-59bdh"] Jun 03 16:21:58.117740 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:58.117713 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ff0c4e_d93c_47b4_ac77_8bc640d4aa6d.slice/crio-4007cf83e767c8fc6dfbc196726458279c462ed46b73e677f2fdbdd7ed59fa86 WatchSource:0}: Error finding container 4007cf83e767c8fc6dfbc196726458279c462ed46b73e677f2fdbdd7ed59fa86: Status 404 returned error can't find the container with id 4007cf83e767c8fc6dfbc196726458279c462ed46b73e677f2fdbdd7ed59fa86 Jun 03 16:21:58.333010 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:58.332970 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m458w"] Jun 03 16:21:58.338085 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:58.338060 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-scd2b"] Jun 03 16:21:58.338629 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:58.338603 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35619cd5_0a7d_4752_bbc3_321dd5e1b1da.slice/crio-686d0d4ec1a27842291c970ab828e4155dd75b6241afc790d0facd3b3c627c75 WatchSource:0}: Error finding container 686d0d4ec1a27842291c970ab828e4155dd75b6241afc790d0facd3b3c627c75: Status 404 returned error can't find the container with id 686d0d4ec1a27842291c970ab828e4155dd75b6241afc790d0facd3b3c627c75 Jun 03 16:21:58.341009 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:21:58.340989 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5834ad_1a0b_4dd4_a05b_c4680b2bb5e8.slice/crio-b5008ed5808e66f3ab9c66b3f8ba32372a6118051575e61285d5544e324b6891 WatchSource:0}: Error finding container b5008ed5808e66f3ab9c66b3f8ba32372a6118051575e61285d5544e324b6891: Status 404 returned error can't find the container with id b5008ed5808e66f3ab9c66b3f8ba32372a6118051575e61285d5544e324b6891 Jun 03 16:21:58.489046 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:58.489008 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-scd2b" event={"ID":"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8","Type":"ContainerStarted","Data":"7b5e257847a99d5d98f599ec088c0e7e63d2fd95c5755c8f8fb1019d3b761779"} Jun 03 16:21:58.489193 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:58.489052 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-scd2b" event={"ID":"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8","Type":"ContainerStarted","Data":"b5008ed5808e66f3ab9c66b3f8ba32372a6118051575e61285d5544e324b6891"} Jun 03 16:21:58.490001 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:58.489964 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-59bdh" event={"ID":"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d","Type":"ContainerStarted","Data":"4007cf83e767c8fc6dfbc196726458279c462ed46b73e677f2fdbdd7ed59fa86"} Jun 03 16:21:58.490960 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:58.490943 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m458w" event={"ID":"35619cd5-0a7d-4752-bbc3-321dd5e1b1da","Type":"ContainerStarted","Data":"686d0d4ec1a27842291c970ab828e4155dd75b6241afc790d0facd3b3c627c75"} Jun 03 16:21:59.308561 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.308492 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:21:59.308561 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.308510 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:21:59.313341 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.312940 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 16:21:59.313341 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.312989 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4xxw6\"" Jun 03 16:21:59.313341 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.313204 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 16:21:59.313341 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.313210 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhlzj\"" Jun 03 16:21:59.313609 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.313494 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 16:21:59.494978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.494950 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-scd2b" event={"ID":"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8","Type":"ContainerStarted","Data":"84eb40a0bda4d0c18ce966b2368bea7d3216dfb14f07f9d932c3c2c001c8cd98"} Jun 03 16:21:59.496272 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:21:59.496251 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-59bdh" event={"ID":"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d","Type":"ContainerStarted","Data":"71696e4f0ff5e80a6da2b0e7d020745d77b88564049dd50b059230310a233087"} Jun 03 16:22:00.500436 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:00.500359 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-59bdh" event={"ID":"e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d","Type":"ContainerStarted","Data":"baa07f5dde2cc2e20fa7d5c41054ebaadc84cecb754528bf9b0056198ecf915d"} Jun 03 16:22:00.500856 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:00.500631 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-59bdh" Jun 03 16:22:00.501835 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:00.501811 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m458w" event={"ID":"35619cd5-0a7d-4752-bbc3-321dd5e1b1da","Type":"ContainerStarted","Data":"e9035daee7a0125d333862dbb82400afe644c58ca01d95eccdee86f7d03959ee"} Jun 03 16:22:00.519512 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:00.519455 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-59bdh" podStartSLOduration=2.24841048 podStartE2EDuration="3.519439705s" podCreationTimestamp="2026-06-03 16:21:57 +0000 UTC" firstStartedPulling="2026-06-03 16:21:58.118987759 +0000 UTC m=+48.383051612" lastFinishedPulling="2026-06-03 16:21:59.39001697 +0000 UTC m=+49.654080837" observedRunningTime="2026-06-03 16:22:00.518781192 +0000 UTC m=+50.782845072" watchObservedRunningTime="2026-06-03 16:22:00.519439705 +0000 UTC m=+50.783503580" Jun 03 16:22:01.041623 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.041573 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m458w" podStartSLOduration=2.335391557 podStartE2EDuration="4.041553025s" podCreationTimestamp="2026-06-03 16:21:57 +0000 UTC" firstStartedPulling="2026-06-03 16:21:58.340311633 +0000 UTC m=+48.604375486" lastFinishedPulling="2026-06-03 16:22:00.046473088 +0000 UTC m=+50.310536954" observedRunningTime="2026-06-03 16:22:00.534199361 +0000 UTC m=+50.798263236" watchObservedRunningTime="2026-06-03 16:22:01.041553025 +0000 UTC m=+51.305616899" Jun 03 16:22:01.041776 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.041704 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-78c59bcf45-zqsg8"] Jun 03 16:22:01.045398 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.045376 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.047974 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.047947 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s5t7g\"" Jun 03 16:22:01.048101 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.047981 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 03 16:22:01.048914 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.048897 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 03 16:22:01.049221 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.049203 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 03 16:22:01.049301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.049287 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 03 16:22:01.049524 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.049407 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 03 16:22:01.049524 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.049475 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 03 16:22:01.049637 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.049563 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 03 16:22:01.056626 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.056608 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c59bcf45-zqsg8"] Jun 03 16:22:01.062212 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.062186 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-oauth-config\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.062312 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.062244 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-config\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.062312 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.062270 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-oauth-serving-cert\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.062450 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.062312 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-serving-cert\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.062450 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.062374 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-service-ca\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.062516 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.062450 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4rv7\" (UniqueName: \"kubernetes.io/projected/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-kube-api-access-c4rv7\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.162929 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.162906 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4rv7\" (UniqueName: \"kubernetes.io/projected/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-kube-api-access-c4rv7\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.163024 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.162939 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-oauth-config\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.163024 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.162966 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-config\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.163024 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.162983 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-oauth-serving-cert\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.163024 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.163015 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-serving-cert\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.163232 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.163031 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-service-ca\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.163795 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.163774 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-oauth-serving-cert\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.163892 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.163801 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-config\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.163892 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.163801 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-service-ca\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.165012 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.164994 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-oauth-config\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.165266 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.165246 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-serving-cert\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.170559 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.170537 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4rv7\" (UniqueName: \"kubernetes.io/projected/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-kube-api-access-c4rv7\") pod \"console-78c59bcf45-zqsg8\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.353989 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.353925 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:01.464044 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.464016 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c59bcf45-zqsg8"] Jun 03 16:22:01.467239 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:01.467209 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1bf43b_9000_4df1_bd11_4a9573e7e5d1.slice/crio-5128dca94b46f0dcae4ecd799488fb66520376bb59406e7ceb3800c6a37e7dd2 WatchSource:0}: Error finding container 5128dca94b46f0dcae4ecd799488fb66520376bb59406e7ceb3800c6a37e7dd2: Status 404 returned error can't find the container with id 5128dca94b46f0dcae4ecd799488fb66520376bb59406e7ceb3800c6a37e7dd2 Jun 03 16:22:01.509137 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.509106 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-scd2b" event={"ID":"4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8","Type":"ContainerStarted","Data":"1a7e2f9b8e8c6ff8eaf236ce96e7ba3f454e88fb6ca17b6683f77d72ef8e39d5"} Jun 03 16:22:01.510292 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.510268 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c59bcf45-zqsg8" event={"ID":"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1","Type":"ContainerStarted","Data":"5128dca94b46f0dcae4ecd799488fb66520376bb59406e7ceb3800c6a37e7dd2"} Jun 03 16:22:01.526876 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:01.526837 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-scd2b" podStartSLOduration=1.8493872649999998 podStartE2EDuration="4.5268263s" podCreationTimestamp="2026-06-03 16:21:57 +0000 UTC" firstStartedPulling="2026-06-03 16:21:58.409422129 +0000 UTC m=+48.673485982" lastFinishedPulling="2026-06-03 16:22:01.08686116 +0000 UTC m=+51.350925017" observedRunningTime="2026-06-03 16:22:01.525974993 +0000 UTC m=+51.790038865" watchObservedRunningTime="2026-06-03 16:22:01.5268263 +0000 UTC m=+51.790890154" Jun 03 16:22:02.109825 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.109795 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr"] Jun 03 16:22:02.141645 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.141615 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr"] Jun 03 16:22:02.141804 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.141696 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" Jun 03 16:22:02.144354 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.144306 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 03 16:22:02.144506 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.144491 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-n7d2b\"" Jun 03 16:22:02.170855 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.170834 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dcfff5a3-832d-4b05-a0f0-acf690d13419-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-zb2kr\" (UID: \"dcfff5a3-832d-4b05-a0f0-acf690d13419\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" Jun 03 16:22:02.271614 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.271570 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dcfff5a3-832d-4b05-a0f0-acf690d13419-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-zb2kr\" (UID: \"dcfff5a3-832d-4b05-a0f0-acf690d13419\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" Jun 03 16:22:02.271786 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:02.271735 2576 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 03 16:22:02.271842 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:02.271801 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcfff5a3-832d-4b05-a0f0-acf690d13419-tls-certificates podName:dcfff5a3-832d-4b05-a0f0-acf690d13419 nodeName:}" failed. No retries permitted until 2026-06-03 16:22:02.77178548 +0000 UTC m=+53.035849336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/dcfff5a3-832d-4b05-a0f0-acf690d13419-tls-certificates") pod "prometheus-operator-admission-webhook-59b7f8756b-zb2kr" (UID: "dcfff5a3-832d-4b05-a0f0-acf690d13419") : secret "prometheus-operator-admission-webhook-tls" not found Jun 03 16:22:02.776147 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.776112 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dcfff5a3-832d-4b05-a0f0-acf690d13419-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-zb2kr\" (UID: \"dcfff5a3-832d-4b05-a0f0-acf690d13419\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" Jun 03 16:22:02.778419 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:02.778402 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dcfff5a3-832d-4b05-a0f0-acf690d13419-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-zb2kr\" (UID: \"dcfff5a3-832d-4b05-a0f0-acf690d13419\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" Jun 03 16:22:03.049806 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:03.049726 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" Jun 03 16:22:03.181659 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:03.181629 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr"] Jun 03 16:22:03.185078 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:03.185051 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfff5a3_832d_4b05_a0f0_acf690d13419.slice/crio-2879e72536e4c3406deb58e40146e9e8ffd3741e1c9140d18917d6c9390601f3 WatchSource:0}: Error finding container 2879e72536e4c3406deb58e40146e9e8ffd3741e1c9140d18917d6c9390601f3: Status 404 returned error can't find the container with id 2879e72536e4c3406deb58e40146e9e8ffd3741e1c9140d18917d6c9390601f3 Jun 03 16:22:03.517893 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:03.517853 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" event={"ID":"dcfff5a3-832d-4b05-a0f0-acf690d13419","Type":"ContainerStarted","Data":"2879e72536e4c3406deb58e40146e9e8ffd3741e1c9140d18917d6c9390601f3"} Jun 03 16:22:06.529081 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:06.529047 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" event={"ID":"dcfff5a3-832d-4b05-a0f0-acf690d13419","Type":"ContainerStarted","Data":"71e4a44fded7be3c4f31e2aa4330253878943e9f1a2b65270023471e684cdd21"} Jun 03 16:22:06.529473 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:06.529270 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" Jun 03 16:22:06.530360 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:06.530334 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c59bcf45-zqsg8" event={"ID":"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1","Type":"ContainerStarted","Data":"0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc"} Jun 03 16:22:06.533891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:06.533873 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" Jun 03 16:22:06.561729 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:06.561696 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-78c59bcf45-zqsg8" podStartSLOduration=1.519880278 podStartE2EDuration="5.561683345s" podCreationTimestamp="2026-06-03 16:22:01 +0000 UTC" firstStartedPulling="2026-06-03 16:22:01.469051717 +0000 UTC m=+51.733115571" lastFinishedPulling="2026-06-03 16:22:05.510854781 +0000 UTC m=+55.774918638" observedRunningTime="2026-06-03 16:22:06.560707312 +0000 UTC m=+56.824771186" watchObservedRunningTime="2026-06-03 16:22:06.561683345 +0000 UTC m=+56.825747565" Jun 03 16:22:06.561834 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:06.561802 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-zb2kr" podStartSLOduration=2.235683414 podStartE2EDuration="4.561798407s" podCreationTimestamp="2026-06-03 16:22:02 +0000 UTC" firstStartedPulling="2026-06-03 16:22:03.187041455 +0000 UTC m=+53.451105308" lastFinishedPulling="2026-06-03 16:22:05.513156448 +0000 UTC m=+55.777220301" observedRunningTime="2026-06-03 16:22:06.544945518 +0000 UTC m=+56.809009392" watchObservedRunningTime="2026-06-03 16:22:06.561798407 +0000 UTC m=+56.825862261" Jun 03 16:22:07.170280 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.170251 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-566kj"] Jun 03 16:22:07.203165 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.203139 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-566kj"] Jun 03 16:22:07.203273 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.203239 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.205891 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.205872 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 03 16:22:07.206101 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.206087 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dpjms\"" Jun 03 16:22:07.206175 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.206101 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 16:22:07.207260 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.207223 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 03 16:22:07.207260 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.207243 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 16:22:07.207260 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.207224 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 16:22:07.237721 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.237700 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5b84d6d-f5sd7"] Jun 03 16:22:07.262467 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.262451 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5b84d6d-f5sd7"] Jun 03 16:22:07.262552 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.262536 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.270238 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.270219 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 03 16:22:07.305866 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.305844 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-oauth-serving-cert\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.305951 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.305875 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.305951 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.305897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-serving-cert\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.305951 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.305911 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9f6n\" (UniqueName: \"kubernetes.io/projected/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-kube-api-access-t9f6n\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.306053 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.305954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.306053 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.305985 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b29281c-7554-4815-a6d7-2f509a99dc6e-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.306053 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.306011 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5krqs\" (UniqueName: \"kubernetes.io/projected/8b29281c-7554-4815-a6d7-2f509a99dc6e-kube-api-access-5krqs\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.306053 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.306031 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-oauth-config\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.306053 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.306047 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-config\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.306243 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.306089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-trusted-ca-bundle\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.306243 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.306128 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-service-ca\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.406729 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.406703 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-serving-cert\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.406845 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.406737 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t9f6n\" (UniqueName: \"kubernetes.io/projected/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-kube-api-access-t9f6n\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.406845 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.406764 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.406845 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.406790 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b29281c-7554-4815-a6d7-2f509a99dc6e-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.406845 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.406819 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5krqs\" (UniqueName: \"kubernetes.io/projected/8b29281c-7554-4815-a6d7-2f509a99dc6e-kube-api-access-5krqs\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.407048 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:07.406844 2576 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 03 16:22:07.407048 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.406850 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-oauth-config\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.407048 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:07.406902 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-tls podName:8b29281c-7554-4815-a6d7-2f509a99dc6e nodeName:}" failed. No retries permitted until 2026-06-03 16:22:07.906883642 +0000 UTC m=+58.170947495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-tls") pod "prometheus-operator-658bcd8bdf-566kj" (UID: "8b29281c-7554-4815-a6d7-2f509a99dc6e") : secret "prometheus-operator-tls" not found Jun 03 16:22:07.407048 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.406935 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-config\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.407048 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.407001 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-trusted-ca-bundle\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.407048 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.407031 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-service-ca\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.407357 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.407060 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-oauth-serving-cert\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.407357 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.407091 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.407661 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.407637 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b29281c-7554-4815-a6d7-2f509a99dc6e-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.407742 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.407663 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-config\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.407857 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.407830 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-service-ca\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.408103 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.408080 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-trusted-ca-bundle\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.408179 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.408166 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-oauth-serving-cert\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.409437 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.409420 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.409631 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.409617 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-serving-cert\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.409670 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.409651 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-oauth-config\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.417365 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.417342 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5krqs\" (UniqueName: \"kubernetes.io/projected/8b29281c-7554-4815-a6d7-2f509a99dc6e-kube-api-access-5krqs\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.417604 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.417590 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9f6n\" (UniqueName: \"kubernetes.io/projected/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-kube-api-access-t9f6n\") pod \"console-5c5b84d6d-f5sd7\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.570803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.570783 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:07.686228 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.686199 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5b84d6d-f5sd7"] Jun 03 16:22:07.689184 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:07.689165 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb92471_62e7_4ec5_a8ed_652f78c9c4b2.slice/crio-eae2f8d286d06caf70073cdcad66e405e34e5119bf594ed0a35bc10b5fc13825 WatchSource:0}: Error finding container eae2f8d286d06caf70073cdcad66e405e34e5119bf594ed0a35bc10b5fc13825: Status 404 returned error can't find the container with id eae2f8d286d06caf70073cdcad66e405e34e5119bf594ed0a35bc10b5fc13825 Jun 03 16:22:07.912135 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.912055 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:07.914245 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:07.914222 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b29281c-7554-4815-a6d7-2f509a99dc6e-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-566kj\" (UID: \"8b29281c-7554-4815-a6d7-2f509a99dc6e\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:08.112333 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:08.112284 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" Jun 03 16:22:08.223946 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:08.223917 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-566kj"] Jun 03 16:22:08.226817 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:08.226793 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b29281c_7554_4815_a6d7_2f509a99dc6e.slice/crio-281877add327ace1fe6bfbb4ac83bcf84b0bfbfd94d258f7cdddab5b4ba94a61 WatchSource:0}: Error finding container 281877add327ace1fe6bfbb4ac83bcf84b0bfbfd94d258f7cdddab5b4ba94a61: Status 404 returned error can't find the container with id 281877add327ace1fe6bfbb4ac83bcf84b0bfbfd94d258f7cdddab5b4ba94a61 Jun 03 16:22:08.537040 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:08.537009 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5b84d6d-f5sd7" event={"ID":"afb92471-62e7-4ec5-a8ed-652f78c9c4b2","Type":"ContainerStarted","Data":"50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6"} Jun 03 16:22:08.537188 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:08.537047 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5b84d6d-f5sd7" event={"ID":"afb92471-62e7-4ec5-a8ed-652f78c9c4b2","Type":"ContainerStarted","Data":"eae2f8d286d06caf70073cdcad66e405e34e5119bf594ed0a35bc10b5fc13825"} Jun 03 16:22:08.538044 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:08.538020 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" event={"ID":"8b29281c-7554-4815-a6d7-2f509a99dc6e","Type":"ContainerStarted","Data":"281877add327ace1fe6bfbb4ac83bcf84b0bfbfd94d258f7cdddab5b4ba94a61"} Jun 03 16:22:08.556131 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:08.556087 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5c5b84d6d-f5sd7" podStartSLOduration=1.556075233 podStartE2EDuration="1.556075233s" podCreationTimestamp="2026-06-03 16:22:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:22:08.555815046 +0000 UTC m=+58.819878920" watchObservedRunningTime="2026-06-03 16:22:08.556075233 +0000 UTC m=+58.820139131" Jun 03 16:22:09.462711 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:09.462682 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-68jfx" Jun 03 16:22:10.512615 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:10.512585 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-59bdh" Jun 03 16:22:10.544605 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:10.544578 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" event={"ID":"8b29281c-7554-4815-a6d7-2f509a99dc6e","Type":"ContainerStarted","Data":"595e057fa80859ef642a0efa9d29ad1a7c2c871a8a0351033cadd481d355d4a4"} Jun 03 16:22:10.544605 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:10.544606 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" event={"ID":"8b29281c-7554-4815-a6d7-2f509a99dc6e","Type":"ContainerStarted","Data":"ef300a7e04145cf654c90dcf8652da9b13d299cf771fec56177d621b8e1736bf"} Jun 03 16:22:10.562616 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:10.562575 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-566kj" podStartSLOduration=2.1818994 podStartE2EDuration="3.56256318s" podCreationTimestamp="2026-06-03 16:22:07 +0000 UTC" firstStartedPulling="2026-06-03 16:22:08.228669122 +0000 UTC m=+58.492732975" lastFinishedPulling="2026-06-03 16:22:09.609332898 +0000 UTC m=+59.873396755" observedRunningTime="2026-06-03 16:22:10.562468261 +0000 UTC m=+60.826532137" watchObservedRunningTime="2026-06-03 16:22:10.56256318 +0000 UTC m=+60.826627053" Jun 03 16:22:11.354282 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:11.354246 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:11.354468 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:11.354291 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:11.358978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:11.358958 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:11.550762 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:11.550731 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:12.640875 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.640842 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-xlzcd"] Jun 03 16:22:12.669387 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.669363 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nt5ph"] Jun 03 16:22:12.669519 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.669506 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.672439 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.672421 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 03 16:22:12.672815 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.672783 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7hvf9\"" Jun 03 16:22:12.672907 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.672892 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 03 16:22:12.689458 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.689438 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-xlzcd"] Jun 03 16:22:12.689458 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.689459 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-sppp4"] Jun 03 16:22:12.689614 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.689578 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.692197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.692179 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 16:22:12.692294 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.692181 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d77tz\"" Jun 03 16:22:12.692294 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.692181 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 16:22:12.692294 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.692217 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 16:22:12.715603 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.715524 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-sppp4"] Jun 03 16:22:12.715682 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.715636 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.718480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.718462 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 03 16:22:12.718602 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.718586 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-c6js4\"" Jun 03 16:22:12.718665 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.718636 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 03 16:22:12.718939 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.718926 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 03 16:22:12.747294 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747275 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.747393 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747308 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.747393 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjp2m\" (UniqueName: \"kubernetes.io/projected/756ce28d-9e95-445e-a661-11f9b38dd228-kube-api-access-wjp2m\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.747393 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747363 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-root\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.747393 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.747551 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.747551 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747411 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d202eb5-629e-464a-bd86-e7810e466d94-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.747551 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747448 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d202eb5-629e-464a-bd86-e7810e466d94-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.747551 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747468 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-accelerators-collector-config\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.747551 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747525 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-wtmp\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.747784 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9bgh\" (UniqueName: \"kubernetes.io/projected/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-kube-api-access-p9bgh\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.747784 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747577 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.747784 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747676 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-sys\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.747784 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv8h7\" (UniqueName: \"kubernetes.io/projected/2d202eb5-629e-464a-bd86-e7810e466d94-kube-api-access-lv8h7\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.747784 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747772 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-tls\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.748002 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-textfile\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.748002 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747826 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/756ce28d-9e95-445e-a661-11f9b38dd228-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.748002 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747865 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-metrics-client-ca\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.748002 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.747888 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.848654 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.848626 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-sys\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.848755 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.848660 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lv8h7\" (UniqueName: \"kubernetes.io/projected/2d202eb5-629e-464a-bd86-e7810e466d94-kube-api-access-lv8h7\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.848755 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.848680 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-tls\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.848863 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.848812 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-sys\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.848898 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.848863 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-textfile\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.848898 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.848891 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/756ce28d-9e95-445e-a661-11f9b38dd228-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.849005 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:12.848988 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 03 16:22:12.849056 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849006 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-metrics-client-ca\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.849056 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849036 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.849056 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:12.849046 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-tls podName:c3a97e66-c7f4-4b85-b7d7-590ef5b3321b nodeName:}" failed. No retries permitted until 2026-06-03 16:22:13.349031182 +0000 UTC m=+63.613095035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-tls") pod "node-exporter-nt5ph" (UID: "c3a97e66-c7f4-4b85-b7d7-590ef5b3321b") : secret "node-exporter-tls" not found Jun 03 16:22:12.849220 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:12.849147 2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 03 16:22:12.849220 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:12.849204 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-tls podName:756ce28d-9e95-445e-a661-11f9b38dd228 nodeName:}" failed. No retries permitted until 2026-06-03 16:22:13.349188888 +0000 UTC m=+63.613252755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-tls") pod "openshift-state-metrics-85658796db-xlzcd" (UID: "756ce28d-9e95-445e-a661-11f9b38dd228") : secret "openshift-state-metrics-tls" not found Jun 03 16:22:12.849315 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849185 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.849315 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849240 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-textfile\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.849315 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849280 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.849487 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849314 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wjp2m\" (UniqueName: \"kubernetes.io/projected/756ce28d-9e95-445e-a661-11f9b38dd228-kube-api-access-wjp2m\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.849487 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849374 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-root\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.849487 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849405 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.849487 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849434 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.849487 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849475 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-root\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.849735 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849602 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/756ce28d-9e95-445e-a661-11f9b38dd228-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.849735 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849481 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d202eb5-629e-464a-bd86-e7810e466d94-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.849834 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849748 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d202eb5-629e-464a-bd86-e7810e466d94-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.849834 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849782 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-metrics-client-ca\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.849834 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849802 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-accelerators-collector-config\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.850076 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.850054 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d202eb5-629e-464a-bd86-e7810e466d94-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.850137 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.850074 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-wtmp\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.850194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.850147 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d202eb5-629e-464a-bd86-e7810e466d94-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.850194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.850166 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.850295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.849838 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-wtmp\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.850295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.850235 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bgh\" (UniqueName: \"kubernetes.io/projected/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-kube-api-access-p9bgh\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.850295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.850262 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.850295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.850286 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-accelerators-collector-config\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.851964 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.851925 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.852055 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.852003 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.852190 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.852171 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.852764 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.852744 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d202eb5-629e-464a-bd86-e7810e466d94-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:12.873633 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.873608 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjp2m\" (UniqueName: \"kubernetes.io/projected/756ce28d-9e95-445e-a661-11f9b38dd228-kube-api-access-wjp2m\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:12.873983 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.873964 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9bgh\" (UniqueName: \"kubernetes.io/projected/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-kube-api-access-p9bgh\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:12.874628 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:12.874606 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv8h7\" (UniqueName: \"kubernetes.io/projected/2d202eb5-629e-464a-bd86-e7810e466d94-kube-api-access-lv8h7\") pod \"kube-state-metrics-8469c486bb-sppp4\" (UID: \"2d202eb5-629e-464a-bd86-e7810e466d94\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:13.023943 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.023911 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" Jun 03 16:22:13.150849 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.150826 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-sppp4"] Jun 03 16:22:13.151951 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:13.151930 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d202eb5_629e_464a_bd86_e7810e466d94.slice/crio-cb6f969492e650b86d16d1a7c6421845bea52f12cbd81c565acf37eae8a6cfc9 WatchSource:0}: Error finding container cb6f969492e650b86d16d1a7c6421845bea52f12cbd81c565acf37eae8a6cfc9: Status 404 returned error can't find the container with id cb6f969492e650b86d16d1a7c6421845bea52f12cbd81c565acf37eae8a6cfc9 Jun 03 16:22:13.354939 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.354875 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-tls\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:13.354939 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.354925 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:13.357005 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.356985 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3a97e66-c7f4-4b85-b7d7-590ef5b3321b-node-exporter-tls\") pod \"node-exporter-nt5ph\" (UID: \"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b\") " pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:13.357199 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.357183 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/756ce28d-9e95-445e-a661-11f9b38dd228-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-xlzcd\" (UID: \"756ce28d-9e95-445e-a661-11f9b38dd228\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:13.553721 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.553680 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" event={"ID":"2d202eb5-629e-464a-bd86-e7810e466d94","Type":"ContainerStarted","Data":"cb6f969492e650b86d16d1a7c6421845bea52f12cbd81c565acf37eae8a6cfc9"} Jun 03 16:22:13.578066 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.578041 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" Jun 03 16:22:13.598683 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.598658 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nt5ph" Jun 03 16:22:13.607678 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:13.607651 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a97e66_c7f4_4b85_b7d7_590ef5b3321b.slice/crio-6c19c2b2df4acea6bbabda3bd3638e5cbbad3559c41d4f2c12dfca9737949f50 WatchSource:0}: Error finding container 6c19c2b2df4acea6bbabda3bd3638e5cbbad3559c41d4f2c12dfca9737949f50: Status 404 returned error can't find the container with id 6c19c2b2df4acea6bbabda3bd3638e5cbbad3559c41d4f2c12dfca9737949f50 Jun 03 16:22:13.707994 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:13.707972 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-xlzcd"] Jun 03 16:22:13.710264 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:13.710239 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756ce28d_9e95_445e_a661_11f9b38dd228.slice/crio-90a12572e89b91dc10ea8d87da44664b78a523f20c3e6ae3e9179a3b5be50fda WatchSource:0}: Error finding container 90a12572e89b91dc10ea8d87da44664b78a523f20c3e6ae3e9179a3b5be50fda: Status 404 returned error can't find the container with id 90a12572e89b91dc10ea8d87da44664b78a523f20c3e6ae3e9179a3b5be50fda Jun 03 16:22:14.558732 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:14.558686 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nt5ph" event={"ID":"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b","Type":"ContainerStarted","Data":"6c19c2b2df4acea6bbabda3bd3638e5cbbad3559c41d4f2c12dfca9737949f50"} Jun 03 16:22:14.560480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:14.560455 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" event={"ID":"756ce28d-9e95-445e-a661-11f9b38dd228","Type":"ContainerStarted","Data":"e75a96a7c074a32cac17c3f5c25630467a775b6a2872f8b14b660b733a0c1ae3"} Jun 03 16:22:14.560594 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:14.560486 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" event={"ID":"756ce28d-9e95-445e-a661-11f9b38dd228","Type":"ContainerStarted","Data":"ba147fd15f00e6e0ebc5f6f9824a6819b5e697e8750020c72e51d14337726668"} Jun 03 16:22:14.560594 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:14.560502 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" event={"ID":"756ce28d-9e95-445e-a661-11f9b38dd228","Type":"ContainerStarted","Data":"90a12572e89b91dc10ea8d87da44664b78a523f20c3e6ae3e9179a3b5be50fda"} Jun 03 16:22:15.564716 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.564681 2576 generic.go:358] "Generic (PLEG): container finished" podID="c3a97e66-c7f4-4b85-b7d7-590ef5b3321b" containerID="c55e6916f9d70581ae37dff8264116a88491c4c4048ed80b25c4b6043c7229b5" exitCode=0 Jun 03 16:22:15.565121 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.564772 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nt5ph" event={"ID":"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b","Type":"ContainerDied","Data":"c55e6916f9d70581ae37dff8264116a88491c4c4048ed80b25c4b6043c7229b5"} Jun 03 16:22:15.566646 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.566618 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" event={"ID":"2d202eb5-629e-464a-bd86-e7810e466d94","Type":"ContainerStarted","Data":"6bebde4f4e7e52bb5de655e59a6b15173eade3e72372466346576695d8e5b832"} Jun 03 16:22:15.566646 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.566643 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" event={"ID":"2d202eb5-629e-464a-bd86-e7810e466d94","Type":"ContainerStarted","Data":"107364f67fe06f5f3045968dad483384add0ae886e3d9ca6746fca314320a455"} Jun 03 16:22:15.566831 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.566658 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" event={"ID":"2d202eb5-629e-464a-bd86-e7810e466d94","Type":"ContainerStarted","Data":"1e9de5d8380a80ba8a47a18cb3e326e57627fa01521a66eb18b606e07fba3b93"} Jun 03 16:22:15.568502 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.568476 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" event={"ID":"756ce28d-9e95-445e-a661-11f9b38dd228","Type":"ContainerStarted","Data":"10ee31edae743d9398169304a4f70357ac6d34a1139e7d67ef7434a8f85e9963"} Jun 03 16:22:15.604815 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.604430 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-sppp4" podStartSLOduration=2.122895362 podStartE2EDuration="3.604415445s" podCreationTimestamp="2026-06-03 16:22:12 +0000 UTC" firstStartedPulling="2026-06-03 16:22:13.153767169 +0000 UTC m=+63.417831025" lastFinishedPulling="2026-06-03 16:22:14.635287254 +0000 UTC m=+64.899351108" observedRunningTime="2026-06-03 16:22:15.602967274 +0000 UTC m=+65.867031148" watchObservedRunningTime="2026-06-03 16:22:15.604415445 +0000 UTC m=+65.868479319" Jun 03 16:22:15.622110 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.622062 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-xlzcd" podStartSLOduration=2.377068753 podStartE2EDuration="3.62205068s" podCreationTimestamp="2026-06-03 16:22:12 +0000 UTC" firstStartedPulling="2026-06-03 16:22:13.85356911 +0000 UTC m=+64.117632984" lastFinishedPulling="2026-06-03 16:22:15.098551056 +0000 UTC m=+65.362614911" observedRunningTime="2026-06-03 16:22:15.620920175 +0000 UTC m=+65.884984074" watchObservedRunningTime="2026-06-03 16:22:15.62205068 +0000 UTC m=+65.886114554" Jun 03 16:22:15.787563 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.787534 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6b4669468d-hvr8h"] Jun 03 16:22:15.789834 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.789815 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.792239 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.792213 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 03 16:22:15.792239 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.792227 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 03 16:22:15.792413 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.792313 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 03 16:22:15.792583 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.792557 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lxjdn\"" Jun 03 16:22:15.792583 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.792576 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8bhf4eihg3602\"" Jun 03 16:22:15.792734 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.792594 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 03 16:22:15.792734 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.792680 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 03 16:22:15.803093 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.803074 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b4669468d-hvr8h"] Jun 03 16:22:15.879724 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.879647 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-tls\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.879724 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.879679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.879724 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.879702 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35443b97-64f9-4797-bb47-81eebf92b93a-metrics-client-ca\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.879985 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.879821 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.879985 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.879862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-grpc-tls\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.879985 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.879888 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cmfr\" (UniqueName: \"kubernetes.io/projected/35443b97-64f9-4797-bb47-81eebf92b93a-kube-api-access-7cmfr\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.879985 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.879932 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.879985 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.879978 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.980607 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980577 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.980607 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980608 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.980792 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980637 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:22:15.980792 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980660 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-tls\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.980792 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980686 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.980792 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980721 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35443b97-64f9-4797-bb47-81eebf92b93a-metrics-client-ca\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.980792 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980764 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:22:15.981035 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980792 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.981035 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980821 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-grpc-tls\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.981035 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.980843 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7cmfr\" (UniqueName: \"kubernetes.io/projected/35443b97-64f9-4797-bb47-81eebf92b93a-kube-api-access-7cmfr\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.981632 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.981581 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35443b97-64f9-4797-bb47-81eebf92b93a-metrics-client-ca\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.983484 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.983458 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 16:22:15.983578 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.983481 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-tls\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.983578 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.983463 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.983578 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.983565 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 16:22:15.983753 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.983736 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-grpc-tls\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.983791 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.983755 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.983791 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.983739 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.984022 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.984003 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/35443b97-64f9-4797-bb47-81eebf92b93a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.988710 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.988691 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cmfr\" (UniqueName: \"kubernetes.io/projected/35443b97-64f9-4797-bb47-81eebf92b93a-kube-api-access-7cmfr\") pod \"thanos-querier-6b4669468d-hvr8h\" (UID: \"35443b97-64f9-4797-bb47-81eebf92b93a\") " pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:15.993296 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.993275 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06952f51-6375-47cd-b827-88ef7b5d38ae-metrics-certs\") pod \"network-metrics-daemon-89gzm\" (UID: \"06952f51-6375-47cd-b827-88ef7b5d38ae\") " pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:22:15.993626 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:15.993612 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 16:22:16.004431 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.004416 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw4xk\" (UniqueName: \"kubernetes.io/projected/b774cdc0-a68b-4b5a-9d9e-ed83bde9884f-kube-api-access-xw4xk\") pod \"network-check-target-2jjqb\" (UID: \"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f\") " pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:22:16.098166 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.098142 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:16.127455 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.123939 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhlzj\"" Jun 03 16:22:16.130016 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.129964 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4xxw6\"" Jun 03 16:22:16.131025 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.131003 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:22:16.138124 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.138101 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89gzm" Jun 03 16:22:16.263484 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.263458 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b4669468d-hvr8h"] Jun 03 16:22:16.267145 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:16.267096 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35443b97_64f9_4797_bb47_81eebf92b93a.slice/crio-390207cc63206135ee06eda949aee89d2024d192d408c78e8ca9ba2ebaa0c497 WatchSource:0}: Error finding container 390207cc63206135ee06eda949aee89d2024d192d408c78e8ca9ba2ebaa0c497: Status 404 returned error can't find the container with id 390207cc63206135ee06eda949aee89d2024d192d408c78e8ca9ba2ebaa0c497 Jun 03 16:22:16.296833 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.296807 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2jjqb"] Jun 03 16:22:16.300375 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:16.300347 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb774cdc0_a68b_4b5a_9d9e_ed83bde9884f.slice/crio-61362d494ba41f712874e7e2780a6cd52ea0afebb6a7464b3f56f0b6b463df18 WatchSource:0}: Error finding container 61362d494ba41f712874e7e2780a6cd52ea0afebb6a7464b3f56f0b6b463df18: Status 404 returned error can't find the container with id 61362d494ba41f712874e7e2780a6cd52ea0afebb6a7464b3f56f0b6b463df18 Jun 03 16:22:16.314712 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.314685 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-89gzm"] Jun 03 16:22:16.317355 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:16.317333 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06952f51_6375_47cd_b827_88ef7b5d38ae.slice/crio-0e0eba40a790fd0a1bcceabb3452db8852054b22195527f871a00a78d5433be6 WatchSource:0}: Error finding container 0e0eba40a790fd0a1bcceabb3452db8852054b22195527f871a00a78d5433be6: Status 404 returned error can't find the container with id 0e0eba40a790fd0a1bcceabb3452db8852054b22195527f871a00a78d5433be6 Jun 03 16:22:16.573793 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.573756 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nt5ph" event={"ID":"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b","Type":"ContainerStarted","Data":"0242334ee631a54867b586bd9601342e111329cdc7e3fcde824047139eeb91d6"} Jun 03 16:22:16.574157 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.573799 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nt5ph" event={"ID":"c3a97e66-c7f4-4b85-b7d7-590ef5b3321b","Type":"ContainerStarted","Data":"61c36504d9c85aca114ad1a104faa1fc064ed2503573f82b216ca47d37cb169f"} Jun 03 16:22:16.574871 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.574844 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2jjqb" event={"ID":"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f","Type":"ContainerStarted","Data":"61362d494ba41f712874e7e2780a6cd52ea0afebb6a7464b3f56f0b6b463df18"} Jun 03 16:22:16.575703 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.575685 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" event={"ID":"35443b97-64f9-4797-bb47-81eebf92b93a","Type":"ContainerStarted","Data":"390207cc63206135ee06eda949aee89d2024d192d408c78e8ca9ba2ebaa0c497"} Jun 03 16:22:16.576551 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.576517 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-89gzm" event={"ID":"06952f51-6375-47cd-b827-88ef7b5d38ae","Type":"ContainerStarted","Data":"0e0eba40a790fd0a1bcceabb3452db8852054b22195527f871a00a78d5433be6"} Jun 03 16:22:16.601257 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:16.601213 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nt5ph" podStartSLOduration=3.574208815 podStartE2EDuration="4.601200195s" podCreationTimestamp="2026-06-03 16:22:12 +0000 UTC" firstStartedPulling="2026-06-03 16:22:13.609361715 +0000 UTC m=+63.873425572" lastFinishedPulling="2026-06-03 16:22:14.636353097 +0000 UTC m=+64.900416952" observedRunningTime="2026-06-03 16:22:16.60051424 +0000 UTC m=+66.864578116" watchObservedRunningTime="2026-06-03 16:22:16.601200195 +0000 UTC m=+66.865264069" Jun 03 16:22:17.114225 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.113511 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c58d6945f-4bc4b"] Jun 03 16:22:17.116188 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.116153 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.119414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.119225 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 03 16:22:17.119788 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.119767 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 03 16:22:17.122117 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.121515 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 03 16:22:17.122117 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.121611 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-atqa9h3qn5cda\"" Jun 03 16:22:17.122117 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.121650 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9wd2n\"" Jun 03 16:22:17.122117 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.121871 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 03 16:22:17.129167 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.129127 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c58d6945f-4bc4b"] Jun 03 16:22:17.191781 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.191703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-secret-metrics-server-tls\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.191781 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.191751 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7abb010c-e3ca-4235-8ac5-b0c6c501b813-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.192021 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.191791 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-secret-metrics-server-client-certs\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.192021 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.191817 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c28b\" (UniqueName: \"kubernetes.io/projected/7abb010c-e3ca-4235-8ac5-b0c6c501b813-kube-api-access-6c28b\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.192021 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.191903 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7abb010c-e3ca-4235-8ac5-b0c6c501b813-audit-log\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.192021 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.191943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-client-ca-bundle\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.192021 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.191979 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7abb010c-e3ca-4235-8ac5-b0c6c501b813-metrics-server-audit-profiles\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.298369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.296551 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-secret-metrics-server-tls\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.298369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.296599 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7abb010c-e3ca-4235-8ac5-b0c6c501b813-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.298369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.296633 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-secret-metrics-server-client-certs\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.298369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.296661 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6c28b\" (UniqueName: \"kubernetes.io/projected/7abb010c-e3ca-4235-8ac5-b0c6c501b813-kube-api-access-6c28b\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.298369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.296751 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7abb010c-e3ca-4235-8ac5-b0c6c501b813-audit-log\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.298369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.296793 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-client-ca-bundle\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.298369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.296858 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7abb010c-e3ca-4235-8ac5-b0c6c501b813-metrics-server-audit-profiles\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.298369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.297982 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7abb010c-e3ca-4235-8ac5-b0c6c501b813-metrics-server-audit-profiles\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.300223 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.299871 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7abb010c-e3ca-4235-8ac5-b0c6c501b813-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.300382 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.300346 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7abb010c-e3ca-4235-8ac5-b0c6c501b813-audit-log\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.304144 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.303278 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-client-ca-bundle\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.304144 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.303961 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-secret-metrics-server-tls\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.306284 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.306241 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7abb010c-e3ca-4235-8ac5-b0c6c501b813-secret-metrics-server-client-certs\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.308917 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.308857 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c28b\" (UniqueName: \"kubernetes.io/projected/7abb010c-e3ca-4235-8ac5-b0c6c501b813-kube-api-access-6c28b\") pod \"metrics-server-5c58d6945f-4bc4b\" (UID: \"7abb010c-e3ca-4235-8ac5-b0c6c501b813\") " pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.339931 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.339905 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl"] Jun 03 16:22:17.343282 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.343261 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" Jun 03 16:22:17.345639 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.345616 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 03 16:22:17.345720 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.345643 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-ml6pg\"" Jun 03 16:22:17.353939 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.353900 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl"] Jun 03 16:22:17.397583 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.397452 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6712278d-f593-49fb-926e-30788a6a7194-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-8ddxl\" (UID: \"6712278d-f593-49fb-926e-30788a6a7194\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" Jun 03 16:22:17.430834 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.430792 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:17.499574 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.498887 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6712278d-f593-49fb-926e-30788a6a7194-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-8ddxl\" (UID: \"6712278d-f593-49fb-926e-30788a6a7194\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" Jun 03 16:22:17.504064 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.504033 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6712278d-f593-49fb-926e-30788a6a7194-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-8ddxl\" (UID: \"6712278d-f593-49fb-926e-30788a6a7194\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" Jun 03 16:22:17.572295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.571596 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:17.572520 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.572486 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:17.581876 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.581167 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:17.583398 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.583300 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-89gzm" event={"ID":"06952f51-6375-47cd-b827-88ef7b5d38ae","Type":"ContainerStarted","Data":"eaf15b89bd046c2c60ee3253d3eb6b9d33a6754f4fb9d317d1817f26c72832f5"} Jun 03 16:22:17.589513 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.589463 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:17.612721 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.612675 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c58d6945f-4bc4b"] Jun 03 16:22:17.617446 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:17.617411 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7abb010c_e3ca_4235_8ac5_b0c6c501b813.slice/crio-947bd476b2c8cbf704b14b091db9a6a85b5e0353ed4a7d88fd570dd4ea7e3a16 WatchSource:0}: Error finding container 947bd476b2c8cbf704b14b091db9a6a85b5e0353ed4a7d88fd570dd4ea7e3a16: Status 404 returned error can't find the container with id 947bd476b2c8cbf704b14b091db9a6a85b5e0353ed4a7d88fd570dd4ea7e3a16 Jun 03 16:22:17.650951 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.650863 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c59bcf45-zqsg8"] Jun 03 16:22:17.658669 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.657652 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" Jun 03 16:22:17.824550 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:17.824501 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl"] Jun 03 16:22:17.828657 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:17.828468 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6712278d_f593_49fb_926e_30788a6a7194.slice/crio-50075f16291e71be41dde8f3867ca94613d1531f8ddccc8401f34d34fa2aba8c WatchSource:0}: Error finding container 50075f16291e71be41dde8f3867ca94613d1531f8ddccc8401f34d34fa2aba8c: Status 404 returned error can't find the container with id 50075f16291e71be41dde8f3867ca94613d1531f8ddccc8401f34d34fa2aba8c Jun 03 16:22:18.038813 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.038783 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b9c977988-n8b75"] Jun 03 16:22:18.041745 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.041719 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.053783 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.053747 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b9c977988-n8b75"] Jun 03 16:22:18.105720 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.105685 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c26r\" (UniqueName: \"kubernetes.io/projected/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-kube-api-access-4c26r\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.105882 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.105757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-config\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.105882 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.105840 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-oauth-config\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.105882 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.105889 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-trusted-ca-bundle\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.106133 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.105909 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-oauth-serving-cert\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.106133 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.105935 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-service-ca\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.106133 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.105957 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-serving-cert\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.207117 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.207078 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-config\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.207287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.207134 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-oauth-config\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.207287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.207179 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-trusted-ca-bundle\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.207287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.207209 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-oauth-serving-cert\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.207287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.207248 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-service-ca\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.207287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.207272 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-serving-cert\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.207592 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.207333 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4c26r\" (UniqueName: \"kubernetes.io/projected/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-kube-api-access-4c26r\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.208000 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.207947 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-config\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.208341 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.208277 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-trusted-ca-bundle\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.208451 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.208411 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-oauth-serving-cert\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.208845 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.208822 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-service-ca\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.210581 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.210558 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-serving-cert\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.210676 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.210592 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-oauth-config\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.217700 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.217662 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c26r\" (UniqueName: \"kubernetes.io/projected/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-kube-api-access-4c26r\") pod \"console-5b9c977988-n8b75\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.354278 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.354187 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:18.587394 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.587351 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" event={"ID":"6712278d-f593-49fb-926e-30788a6a7194","Type":"ContainerStarted","Data":"50075f16291e71be41dde8f3867ca94613d1531f8ddccc8401f34d34fa2aba8c"} Jun 03 16:22:18.588609 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.588584 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" event={"ID":"7abb010c-e3ca-4235-8ac5-b0c6c501b813","Type":"ContainerStarted","Data":"947bd476b2c8cbf704b14b091db9a6a85b5e0353ed4a7d88fd570dd4ea7e3a16"} Jun 03 16:22:18.590307 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.590278 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-89gzm" event={"ID":"06952f51-6375-47cd-b827-88ef7b5d38ae","Type":"ContainerStarted","Data":"5d665b52aa3ca82a790bee9f723ae86427791c2d876db0bcae6c6a8cf7eed330"} Jun 03 16:22:18.611906 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:18.611821 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-89gzm" podStartSLOduration=67.477249444 podStartE2EDuration="1m8.611807083s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:22:16.319050689 +0000 UTC m=+66.583114542" lastFinishedPulling="2026-06-03 16:22:17.453608312 +0000 UTC m=+67.717672181" observedRunningTime="2026-06-03 16:22:18.610356233 +0000 UTC m=+68.874420104" watchObservedRunningTime="2026-06-03 16:22:18.611807083 +0000 UTC m=+68.875870959" Jun 03 16:22:20.582237 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.582191 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b9c977988-n8b75"] Jun 03 16:22:20.589922 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:22:20.589900 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cbca0c6_3e3b_462e_8fc7_3ff14ab325e7.slice/crio-c758b6f360d2ecb96e6a9ad4c3a38eb05a3549c4d5b011a9df061c7d465b0343 WatchSource:0}: Error finding container c758b6f360d2ecb96e6a9ad4c3a38eb05a3549c4d5b011a9df061c7d465b0343: Status 404 returned error can't find the container with id c758b6f360d2ecb96e6a9ad4c3a38eb05a3549c4d5b011a9df061c7d465b0343 Jun 03 16:22:20.599309 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.599261 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" event={"ID":"35443b97-64f9-4797-bb47-81eebf92b93a","Type":"ContainerStarted","Data":"4e6628d562685438638a4c35a41f46a3697257247ce0786e12fcddc7499ce3bf"} Jun 03 16:22:20.600615 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.600592 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" event={"ID":"6712278d-f593-49fb-926e-30788a6a7194","Type":"ContainerStarted","Data":"c9dac0be4ccae70f245a7c6d8917d9a890e34f08826ba2adb4e06cffc13bca4a"} Jun 03 16:22:20.600863 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.600845 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" Jun 03 16:22:20.602315 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.602248 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" event={"ID":"7abb010c-e3ca-4235-8ac5-b0c6c501b813","Type":"ContainerStarted","Data":"18efe010df509e5ab543051c5a45e6be8db76da25bad56133718b2db798747c4"} Jun 03 16:22:20.604048 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.604029 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9c977988-n8b75" event={"ID":"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7","Type":"ContainerStarted","Data":"c758b6f360d2ecb96e6a9ad4c3a38eb05a3549c4d5b011a9df061c7d465b0343"} Jun 03 16:22:20.606049 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.605663 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2jjqb" event={"ID":"b774cdc0-a68b-4b5a-9d9e-ed83bde9884f","Type":"ContainerStarted","Data":"b149f416dbad45474202284f964c544612960883449172c5b65b770ee65ec70b"} Jun 03 16:22:20.606049 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.605831 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:22:20.606290 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.606274 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" Jun 03 16:22:20.627125 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.627078 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-8ddxl" podStartSLOduration=1.027003726 podStartE2EDuration="3.627063914s" podCreationTimestamp="2026-06-03 16:22:17 +0000 UTC" firstStartedPulling="2026-06-03 16:22:17.831667294 +0000 UTC m=+68.095731150" lastFinishedPulling="2026-06-03 16:22:20.431727466 +0000 UTC m=+70.695791338" observedRunningTime="2026-06-03 16:22:20.623737257 +0000 UTC m=+70.887801133" watchObservedRunningTime="2026-06-03 16:22:20.627063914 +0000 UTC m=+70.891127790" Jun 03 16:22:20.642475 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.641647 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2jjqb" podStartSLOduration=66.519795215 podStartE2EDuration="1m10.641630743s" podCreationTimestamp="2026-06-03 16:21:10 +0000 UTC" firstStartedPulling="2026-06-03 16:22:16.302215802 +0000 UTC m=+66.566279655" lastFinishedPulling="2026-06-03 16:22:20.424051326 +0000 UTC m=+70.688115183" observedRunningTime="2026-06-03 16:22:20.640907195 +0000 UTC m=+70.904971081" watchObservedRunningTime="2026-06-03 16:22:20.641630743 +0000 UTC m=+70.905694619" Jun 03 16:22:20.679542 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:20.679408 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" podStartSLOduration=0.870220265 podStartE2EDuration="3.679368832s" podCreationTimestamp="2026-06-03 16:22:17 +0000 UTC" firstStartedPulling="2026-06-03 16:22:17.620460419 +0000 UTC m=+67.884524278" lastFinishedPulling="2026-06-03 16:22:20.42960898 +0000 UTC m=+70.693672845" observedRunningTime="2026-06-03 16:22:20.678798349 +0000 UTC m=+70.942862237" watchObservedRunningTime="2026-06-03 16:22:20.679368832 +0000 UTC m=+70.943432706" Jun 03 16:22:21.610497 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:21.610472 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" event={"ID":"35443b97-64f9-4797-bb47-81eebf92b93a","Type":"ContainerStarted","Data":"2ace440c9d0941c1573f92255832282833b0651968c8e3b29d157bcd8c7d28be"} Jun 03 16:22:21.610806 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:21.610512 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" event={"ID":"35443b97-64f9-4797-bb47-81eebf92b93a","Type":"ContainerStarted","Data":"80a23f2d47e81a08c6fab844800e1340bf2acaa662d7b17f6a7881b4c5458caa"} Jun 03 16:22:21.611954 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:21.611925 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9c977988-n8b75" event={"ID":"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7","Type":"ContainerStarted","Data":"803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b"} Jun 03 16:22:21.633293 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:21.633243 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5b9c977988-n8b75" podStartSLOduration=3.633227009 podStartE2EDuration="3.633227009s" podCreationTimestamp="2026-06-03 16:22:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:22:21.631373251 +0000 UTC m=+71.895437125" watchObservedRunningTime="2026-06-03 16:22:21.633227009 +0000 UTC m=+71.897290885" Jun 03 16:22:22.617528 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:22.617495 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" event={"ID":"35443b97-64f9-4797-bb47-81eebf92b93a","Type":"ContainerStarted","Data":"e9ef5506537e0760ba167b32cb02655a7636d37499e01f31843350bf93895352"} Jun 03 16:22:22.617528 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:22.617529 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" event={"ID":"35443b97-64f9-4797-bb47-81eebf92b93a","Type":"ContainerStarted","Data":"35ac4efc0b115baf1bca681619d7523c6d177730643499ed017c2c5afe0f4524"} Jun 03 16:22:22.617528 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:22.617538 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" event={"ID":"35443b97-64f9-4797-bb47-81eebf92b93a","Type":"ContainerStarted","Data":"3c8e707ed386ad7cc0ba8c84aae934c13c71f9b31b6814139ffc6516c9a5a75e"} Jun 03 16:22:22.645515 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:22.645473 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" podStartSLOduration=2.355895818 podStartE2EDuration="7.645461307s" podCreationTimestamp="2026-06-03 16:22:15 +0000 UTC" firstStartedPulling="2026-06-03 16:22:16.269370727 +0000 UTC m=+66.533434581" lastFinishedPulling="2026-06-03 16:22:21.558936201 +0000 UTC m=+71.823000070" observedRunningTime="2026-06-03 16:22:22.643750863 +0000 UTC m=+72.907814740" watchObservedRunningTime="2026-06-03 16:22:22.645461307 +0000 UTC m=+72.909525181" Jun 03 16:22:23.620039 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:23.620009 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:28.355026 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:28.354985 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:28.355026 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:28.355030 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:28.359743 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:28.359722 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:28.640034 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:28.639961 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:22:28.712007 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:28.711979 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5b84d6d-f5sd7"] Jun 03 16:22:29.629294 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:29.629268 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6b4669468d-hvr8h" Jun 03 16:22:37.431214 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:37.431179 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:37.431214 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:37.431223 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:42.688890 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:42.688850 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-78c59bcf45-zqsg8" podUID="9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" containerName="console" containerID="cri-o://0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc" gracePeriod=15 Jun 03 16:22:42.928133 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:42.928113 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78c59bcf45-zqsg8_9a1bf43b-9000-4df1-bd11-4a9573e7e5d1/console/0.log" Jun 03 16:22:42.928244 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:42.928181 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:43.117161 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117137 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4rv7\" (UniqueName: \"kubernetes.io/projected/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-kube-api-access-c4rv7\") pod \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " Jun 03 16:22:43.117311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117186 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-oauth-config\") pod \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " Jun 03 16:22:43.117311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117231 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-serving-cert\") pod \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " Jun 03 16:22:43.117311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117276 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-config\") pod \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " Jun 03 16:22:43.117483 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117312 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-service-ca\") pod \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " Jun 03 16:22:43.117483 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117362 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-oauth-serving-cert\") pod \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\" (UID: \"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1\") " Jun 03 16:22:43.117739 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117705 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" (UID: "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:22:43.117851 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117714 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-config" (OuterVolumeSpecName: "console-config") pod "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" (UID: "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:22:43.117851 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.117753 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" (UID: "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:22:43.119380 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.119362 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" (UID: "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:22:43.119486 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.119465 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" (UID: "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:22:43.119553 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.119510 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-kube-api-access-c4rv7" (OuterVolumeSpecName: "kube-api-access-c4rv7") pod "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" (UID: "9a1bf43b-9000-4df1-bd11-4a9573e7e5d1"). InnerVolumeSpecName "kube-api-access-c4rv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:22:43.218679 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.218641 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-service-ca\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:43.218679 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.218677 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-oauth-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:43.218679 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.218687 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4rv7\" (UniqueName: \"kubernetes.io/projected/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-kube-api-access-c4rv7\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:43.218904 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.218697 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-oauth-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:43.218904 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.218707 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:43.218904 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.218715 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1-console-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:43.676053 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.676025 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78c59bcf45-zqsg8_9a1bf43b-9000-4df1-bd11-4a9573e7e5d1/console/0.log" Jun 03 16:22:43.676218 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.676061 2576 generic.go:358] "Generic (PLEG): container finished" podID="9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" containerID="0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc" exitCode=2 Jun 03 16:22:43.676218 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.676147 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c59bcf45-zqsg8" Jun 03 16:22:43.676218 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.676158 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c59bcf45-zqsg8" event={"ID":"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1","Type":"ContainerDied","Data":"0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc"} Jun 03 16:22:43.676218 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.676190 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c59bcf45-zqsg8" event={"ID":"9a1bf43b-9000-4df1-bd11-4a9573e7e5d1","Type":"ContainerDied","Data":"5128dca94b46f0dcae4ecd799488fb66520376bb59406e7ceb3800c6a37e7dd2"} Jun 03 16:22:43.676218 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.676205 2576 scope.go:117] "RemoveContainer" containerID="0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc" Jun 03 16:22:43.684602 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.684585 2576 scope.go:117] "RemoveContainer" containerID="0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc" Jun 03 16:22:43.684888 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:43.684868 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc\": container with ID starting with 0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc not found: ID does not exist" containerID="0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc" Jun 03 16:22:43.684937 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.684896 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc"} err="failed to get container status \"0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc\": rpc error: code = NotFound desc = could not find container \"0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc\": container with ID starting with 0488e63ebcbc15e1071f904f9cbe2e969f352217a38fddb220ba24f0cc185cdc not found: ID does not exist" Jun 03 16:22:43.697829 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.697809 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c59bcf45-zqsg8"] Jun 03 16:22:43.701390 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:43.701369 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78c59bcf45-zqsg8"] Jun 03 16:22:44.313197 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:44.313163 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" path="/var/lib/kubelet/pods/9a1bf43b-9000-4df1-bd11-4a9573e7e5d1/volumes" Jun 03 16:22:51.614231 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:51.614203 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2jjqb" Jun 03 16:22:53.731173 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:53.731114 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5c5b84d6d-f5sd7" podUID="afb92471-62e7-4ec5-a8ed-652f78c9c4b2" containerName="console" containerID="cri-o://50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6" gracePeriod=15 Jun 03 16:22:53.994568 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:53.994547 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c5b84d6d-f5sd7_afb92471-62e7-4ec5-a8ed-652f78c9c4b2/console/0.log" Jun 03 16:22:53.994675 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:53.994604 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:54.086434 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086406 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-service-ca\") pod \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " Jun 03 16:22:54.086586 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086441 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9f6n\" (UniqueName: \"kubernetes.io/projected/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-kube-api-access-t9f6n\") pod \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " Jun 03 16:22:54.086586 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086462 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-oauth-config\") pod \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " Jun 03 16:22:54.086586 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086496 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-config\") pod \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " Jun 03 16:22:54.086586 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086527 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-serving-cert\") pod \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " Jun 03 16:22:54.086586 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086554 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-trusted-ca-bundle\") pod \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " Jun 03 16:22:54.086843 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086620 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-oauth-serving-cert\") pod \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\" (UID: \"afb92471-62e7-4ec5-a8ed-652f78c9c4b2\") " Jun 03 16:22:54.086843 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086821 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-service-ca" (OuterVolumeSpecName: "service-ca") pod "afb92471-62e7-4ec5-a8ed-652f78c9c4b2" (UID: "afb92471-62e7-4ec5-a8ed-652f78c9c4b2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:22:54.086952 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.086921 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-config" (OuterVolumeSpecName: "console-config") pod "afb92471-62e7-4ec5-a8ed-652f78c9c4b2" (UID: "afb92471-62e7-4ec5-a8ed-652f78c9c4b2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:22:54.087111 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.087085 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "afb92471-62e7-4ec5-a8ed-652f78c9c4b2" (UID: "afb92471-62e7-4ec5-a8ed-652f78c9c4b2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:22:54.087230 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.087208 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "afb92471-62e7-4ec5-a8ed-652f78c9c4b2" (UID: "afb92471-62e7-4ec5-a8ed-652f78c9c4b2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:22:54.088649 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.088622 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "afb92471-62e7-4ec5-a8ed-652f78c9c4b2" (UID: "afb92471-62e7-4ec5-a8ed-652f78c9c4b2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:22:54.088743 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.088692 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "afb92471-62e7-4ec5-a8ed-652f78c9c4b2" (UID: "afb92471-62e7-4ec5-a8ed-652f78c9c4b2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:22:54.088784 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.088755 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-kube-api-access-t9f6n" (OuterVolumeSpecName: "kube-api-access-t9f6n") pod "afb92471-62e7-4ec5-a8ed-652f78c9c4b2" (UID: "afb92471-62e7-4ec5-a8ed-652f78c9c4b2"). InnerVolumeSpecName "kube-api-access-t9f6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:22:54.187669 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.187642 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-oauth-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:54.187669 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.187665 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-service-ca\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:54.187803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.187674 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9f6n\" (UniqueName: \"kubernetes.io/projected/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-kube-api-access-t9f6n\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:54.187803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.187684 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-oauth-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:54.187803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.187692 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:54.187803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.187701 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-console-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:54.187803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.187709 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afb92471-62e7-4ec5-a8ed-652f78c9c4b2-trusted-ca-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:22:54.706561 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.706490 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c5b84d6d-f5sd7_afb92471-62e7-4ec5-a8ed-652f78c9c4b2/console/0.log" Jun 03 16:22:54.706561 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.706527 2576 generic.go:358] "Generic (PLEG): container finished" podID="afb92471-62e7-4ec5-a8ed-652f78c9c4b2" containerID="50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6" exitCode=2 Jun 03 16:22:54.706749 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.706588 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5b84d6d-f5sd7" Jun 03 16:22:54.706749 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.706591 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5b84d6d-f5sd7" event={"ID":"afb92471-62e7-4ec5-a8ed-652f78c9c4b2","Type":"ContainerDied","Data":"50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6"} Jun 03 16:22:54.706749 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.706623 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5b84d6d-f5sd7" event={"ID":"afb92471-62e7-4ec5-a8ed-652f78c9c4b2","Type":"ContainerDied","Data":"eae2f8d286d06caf70073cdcad66e405e34e5119bf594ed0a35bc10b5fc13825"} Jun 03 16:22:54.706749 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.706636 2576 scope.go:117] "RemoveContainer" containerID="50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6" Jun 03 16:22:54.713938 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.713919 2576 scope.go:117] "RemoveContainer" containerID="50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6" Jun 03 16:22:54.714215 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:22:54.714185 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6\": container with ID starting with 50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6 not found: ID does not exist" containerID="50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6" Jun 03 16:22:54.714304 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.714224 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6"} err="failed to get container status \"50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6\": rpc error: code = NotFound desc = could not find container \"50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6\": container with ID starting with 50867053da4d32192109d7b19cbd29dac2cfba442c369ee42695e566a2e4a0e6 not found: ID does not exist" Jun 03 16:22:54.722512 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.722492 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5b84d6d-f5sd7"] Jun 03 16:22:54.727047 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:54.727024 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5b84d6d-f5sd7"] Jun 03 16:22:56.312618 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:56.312586 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afb92471-62e7-4ec5-a8ed-652f78c9c4b2" path="/var/lib/kubelet/pods/afb92471-62e7-4ec5-a8ed-652f78c9c4b2/volumes" Jun 03 16:22:57.436856 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:57.436821 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:22:57.440518 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:22:57.440501 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c58d6945f-4bc4b" Jun 03 16:23:37.038901 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.038866 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-77d48fbc65-nkxr5"] Jun 03 16:23:37.039968 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.039948 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" containerName="console" Jun 03 16:23:37.040087 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.040076 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" containerName="console" Jun 03 16:23:37.040174 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.040164 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afb92471-62e7-4ec5-a8ed-652f78c9c4b2" containerName="console" Jun 03 16:23:37.040247 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.040239 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb92471-62e7-4ec5-a8ed-652f78c9c4b2" containerName="console" Jun 03 16:23:37.040417 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.040393 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="afb92471-62e7-4ec5-a8ed-652f78c9c4b2" containerName="console" Jun 03 16:23:37.040417 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.040411 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="9a1bf43b-9000-4df1-bd11-4a9573e7e5d1" containerName="console" Jun 03 16:23:37.043414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.043399 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.046912 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.046871 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 03 16:23:37.047015 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.046923 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 03 16:23:37.047015 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.046972 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 03 16:23:37.047015 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.046973 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 03 16:23:37.047195 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.047182 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 03 16:23:37.047418 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.047398 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qvwwt\"" Jun 03 16:23:37.054667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.053878 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77d48fbc65-nkxr5"] Jun 03 16:23:37.056277 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.056243 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 03 16:23:37.109566 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.109534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-secret-telemeter-client\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.109716 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.109574 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-telemeter-client-tls\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.109716 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.109596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhqpg\" (UniqueName: \"kubernetes.io/projected/bb2d28b8-a41f-4014-a23d-cce589335ff3-kube-api-access-jhqpg\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.109716 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.109666 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-federate-client-tls\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.109816 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.109722 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.109816 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.109757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-serving-certs-ca-bundle\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.109816 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.109777 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.109905 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.109819 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-metrics-client-ca\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.210936 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.210901 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-serving-certs-ca-bundle\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.210936 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.210937 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211143 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.210955 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-metrics-client-ca\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211143 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.211085 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-secret-telemeter-client\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211242 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.211142 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-telemeter-client-tls\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211242 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.211165 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jhqpg\" (UniqueName: \"kubernetes.io/projected/bb2d28b8-a41f-4014-a23d-cce589335ff3-kube-api-access-jhqpg\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211242 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.211203 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-federate-client-tls\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211412 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.211364 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211740 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.211709 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-metrics-client-ca\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211866 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.211781 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-serving-certs-ca-bundle\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.211985 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.211964 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2d28b8-a41f-4014-a23d-cce589335ff3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.213708 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.213686 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.213793 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.213714 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-secret-telemeter-client\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.213793 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.213750 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-federate-client-tls\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.213861 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.213840 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bb2d28b8-a41f-4014-a23d-cce589335ff3-telemeter-client-tls\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.220422 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.220401 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhqpg\" (UniqueName: \"kubernetes.io/projected/bb2d28b8-a41f-4014-a23d-cce589335ff3-kube-api-access-jhqpg\") pod \"telemeter-client-77d48fbc65-nkxr5\" (UID: \"bb2d28b8-a41f-4014-a23d-cce589335ff3\") " pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.359489 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.359391 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" Jun 03 16:23:37.489464 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.489269 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77d48fbc65-nkxr5"] Jun 03 16:23:37.493859 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:23:37.493830 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2d28b8_a41f_4014_a23d_cce589335ff3.slice/crio-b3dcb7b507425d716501df9f1a3c26093edaea4b91c752702aa808ac2a41f9e7 WatchSource:0}: Error finding container b3dcb7b507425d716501df9f1a3c26093edaea4b91c752702aa808ac2a41f9e7: Status 404 returned error can't find the container with id b3dcb7b507425d716501df9f1a3c26093edaea4b91c752702aa808ac2a41f9e7 Jun 03 16:23:37.824025 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:37.823991 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" event={"ID":"bb2d28b8-a41f-4014-a23d-cce589335ff3","Type":"ContainerStarted","Data":"b3dcb7b507425d716501df9f1a3c26093edaea4b91c752702aa808ac2a41f9e7"} Jun 03 16:23:39.833418 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:39.833386 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" event={"ID":"bb2d28b8-a41f-4014-a23d-cce589335ff3","Type":"ContainerStarted","Data":"f1eff3ae881150ef8dbaed04d6ef665446278afc7b2a89e56457fe85e4659541"} Jun 03 16:23:40.838443 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:40.838405 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" event={"ID":"bb2d28b8-a41f-4014-a23d-cce589335ff3","Type":"ContainerStarted","Data":"bca8e4cf0cafcc2bbdcbc3d7a946d33197f1662b8957eb154989c3d3e03ce81b"} Jun 03 16:23:40.838773 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:40.838449 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" event={"ID":"bb2d28b8-a41f-4014-a23d-cce589335ff3","Type":"ContainerStarted","Data":"7fa576131b4c4b1cdef98fe93e342e2d47955704a26cf790cc05f52a535e695b"} Jun 03 16:23:40.861342 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:40.861271 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-77d48fbc65-nkxr5" podStartSLOduration=0.859743479 podStartE2EDuration="3.861255988s" podCreationTimestamp="2026-06-03 16:23:37 +0000 UTC" firstStartedPulling="2026-06-03 16:23:37.495102497 +0000 UTC m=+147.759166353" lastFinishedPulling="2026-06-03 16:23:40.496615008 +0000 UTC m=+150.760678862" observedRunningTime="2026-06-03 16:23:40.859089725 +0000 UTC m=+151.123153598" watchObservedRunningTime="2026-06-03 16:23:40.861255988 +0000 UTC m=+151.125319861" Jun 03 16:23:41.578431 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.578398 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ff79f989c-d4sh6"] Jun 03 16:23:41.580411 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.580395 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.591264 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.591240 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ff79f989c-d4sh6"] Jun 03 16:23:41.648463 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.648425 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-oauth-config\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.648625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.648529 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-service-ca\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.648625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.648567 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-oauth-serving-cert\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.648625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.648597 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-trusted-ca-bundle\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.648743 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.648650 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-serving-cert\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.648743 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.648676 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-console-config\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.648743 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.648692 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fczrj\" (UniqueName: \"kubernetes.io/projected/9940a591-cc0e-4561-8037-b890e648f56c-kube-api-access-fczrj\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.749457 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.749420 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-oauth-config\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.749650 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.749473 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-service-ca\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.749650 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.749498 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-oauth-serving-cert\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.749650 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.749532 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-trusted-ca-bundle\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.749650 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.749576 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-serving-cert\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.749650 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.749602 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-console-config\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.749650 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.749622 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fczrj\" (UniqueName: \"kubernetes.io/projected/9940a591-cc0e-4561-8037-b890e648f56c-kube-api-access-fczrj\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.750343 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.750305 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-oauth-serving-cert\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.750456 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.750408 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-console-config\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.750456 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.750412 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-service-ca\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.750580 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.750507 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-trusted-ca-bundle\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.751968 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.751947 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-oauth-config\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.752116 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.752097 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-serving-cert\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.757904 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.757885 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fczrj\" (UniqueName: \"kubernetes.io/projected/9940a591-cc0e-4561-8037-b890e648f56c-kube-api-access-fczrj\") pod \"console-7ff79f989c-d4sh6\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:41.889270 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:41.889175 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:42.015887 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:42.015852 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ff79f989c-d4sh6"] Jun 03 16:23:42.019061 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:23:42.019035 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9940a591_cc0e_4561_8037_b890e648f56c.slice/crio-658b5ab335ed9d5e56cdb63eaf1eecdb6b9b4b840ca341bb28947e76ced6050b WatchSource:0}: Error finding container 658b5ab335ed9d5e56cdb63eaf1eecdb6b9b4b840ca341bb28947e76ced6050b: Status 404 returned error can't find the container with id 658b5ab335ed9d5e56cdb63eaf1eecdb6b9b4b840ca341bb28947e76ced6050b Jun 03 16:23:42.845980 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:42.845937 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff79f989c-d4sh6" event={"ID":"9940a591-cc0e-4561-8037-b890e648f56c","Type":"ContainerStarted","Data":"7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6"} Jun 03 16:23:42.845980 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:42.845980 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff79f989c-d4sh6" event={"ID":"9940a591-cc0e-4561-8037-b890e648f56c","Type":"ContainerStarted","Data":"658b5ab335ed9d5e56cdb63eaf1eecdb6b9b4b840ca341bb28947e76ced6050b"} Jun 03 16:23:42.866809 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:42.866762 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7ff79f989c-d4sh6" podStartSLOduration=1.866748411 podStartE2EDuration="1.866748411s" podCreationTimestamp="2026-06-03 16:23:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:23:42.864480969 +0000 UTC m=+153.128544837" watchObservedRunningTime="2026-06-03 16:23:42.866748411 +0000 UTC m=+153.130812286" Jun 03 16:23:51.889969 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:51.889886 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:51.889969 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:51.889951 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:51.894637 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:51.894615 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:52.877986 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:52.877962 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:23:52.937316 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:52.937286 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b9c977988-n8b75"] Jun 03 16:23:53.040659 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.040627 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cb5db9c64-2gl4t"] Jun 03 16:23:53.045004 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.044986 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.054924 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.054897 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cb5db9c64-2gl4t"] Jun 03 16:23:53.141854 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.141756 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-trusted-ca-bundle\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.141854 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.141797 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-serving-cert\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.141854 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.141848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s4hn\" (UniqueName: \"kubernetes.io/projected/068e2ddc-c216-4d70-a8ab-397efbead753-kube-api-access-5s4hn\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.142084 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.141886 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-console-config\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.142084 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.141928 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-oauth-config\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.142084 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.141947 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-oauth-serving-cert\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.142084 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.141963 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-service-ca\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.242439 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.242402 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-oauth-config\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.242622 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.242436 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-oauth-serving-cert\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.242622 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.242472 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-service-ca\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.242622 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.242534 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-trusted-ca-bundle\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.242622 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.242561 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-serving-cert\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.242622 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.242609 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5s4hn\" (UniqueName: \"kubernetes.io/projected/068e2ddc-c216-4d70-a8ab-397efbead753-kube-api-access-5s4hn\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.242869 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.242642 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-console-config\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.243309 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.243282 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-oauth-serving-cert\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.243456 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.243354 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-service-ca\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.243456 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.243385 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-console-config\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.243601 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.243454 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-trusted-ca-bundle\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.245066 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.245046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-serving-cert\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.245145 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.245064 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-oauth-config\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.251974 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.251952 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s4hn\" (UniqueName: \"kubernetes.io/projected/068e2ddc-c216-4d70-a8ab-397efbead753-kube-api-access-5s4hn\") pod \"console-7cb5db9c64-2gl4t\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.354847 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.354809 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:23:53.479812 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.479765 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cb5db9c64-2gl4t"] Jun 03 16:23:53.481800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:23:53.481780 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068e2ddc_c216_4d70_a8ab_397efbead753.slice/crio-ef87f857978f24b792b505483fb0877e04f5ee005a5051237347b6311ca85b4e WatchSource:0}: Error finding container ef87f857978f24b792b505483fb0877e04f5ee005a5051237347b6311ca85b4e: Status 404 returned error can't find the container with id ef87f857978f24b792b505483fb0877e04f5ee005a5051237347b6311ca85b4e Jun 03 16:23:53.878696 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.878659 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb5db9c64-2gl4t" event={"ID":"068e2ddc-c216-4d70-a8ab-397efbead753","Type":"ContainerStarted","Data":"db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5"} Jun 03 16:23:53.878696 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.878699 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb5db9c64-2gl4t" event={"ID":"068e2ddc-c216-4d70-a8ab-397efbead753","Type":"ContainerStarted","Data":"ef87f857978f24b792b505483fb0877e04f5ee005a5051237347b6311ca85b4e"} Jun 03 16:23:53.900053 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:23:53.900008 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7cb5db9c64-2gl4t" podStartSLOduration=0.899992239 podStartE2EDuration="899.992239ms" podCreationTimestamp="2026-06-03 16:23:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:23:53.89765096 +0000 UTC m=+164.161714831" watchObservedRunningTime="2026-06-03 16:23:53.899992239 +0000 UTC m=+164.164056114" Jun 03 16:24:03.355414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:03.355377 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:24:03.355414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:03.355422 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:24:03.360101 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:03.360073 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:24:03.916161 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:03.916130 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:24:03.968219 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:03.968186 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ff79f989c-d4sh6"] Jun 03 16:24:17.957064 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:17.957003 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5b9c977988-n8b75" podUID="6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" containerName="console" containerID="cri-o://803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b" gracePeriod=15 Jun 03 16:24:18.190174 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.190154 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b9c977988-n8b75_6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7/console/0.log" Jun 03 16:24:18.190285 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.190216 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:24:18.245127 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245056 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-serving-cert\") pod \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " Jun 03 16:24:18.245127 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245107 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-oauth-serving-cert\") pod \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " Jun 03 16:24:18.245311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245136 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-oauth-config\") pod \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " Jun 03 16:24:18.245311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245163 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-trusted-ca-bundle\") pod \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " Jun 03 16:24:18.245311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245193 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c26r\" (UniqueName: \"kubernetes.io/projected/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-kube-api-access-4c26r\") pod \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " Jun 03 16:24:18.245311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245231 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-service-ca\") pod \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " Jun 03 16:24:18.245311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245298 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-config\") pod \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\" (UID: \"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7\") " Jun 03 16:24:18.245661 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245537 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" (UID: "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:24:18.245661 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245609 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" (UID: "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:24:18.245839 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245789 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-service-ca" (OuterVolumeSpecName: "service-ca") pod "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" (UID: "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:24:18.245921 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.245897 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-config" (OuterVolumeSpecName: "console-config") pod "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" (UID: "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:24:18.247195 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.247173 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" (UID: "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:24:18.247260 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.247228 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" (UID: "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:24:18.247315 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.247290 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-kube-api-access-4c26r" (OuterVolumeSpecName: "kube-api-access-4c26r") pod "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" (UID: "6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7"). InnerVolumeSpecName "kube-api-access-4c26r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:24:18.346126 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.346100 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:18.346126 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.346122 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:18.346279 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.346132 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-oauth-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:18.346279 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.346141 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-console-oauth-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:18.346279 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.346149 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-trusted-ca-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:18.346279 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.346157 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c26r\" (UniqueName: \"kubernetes.io/projected/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-kube-api-access-4c26r\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:18.346279 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.346168 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7-service-ca\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:18.953075 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.953049 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b9c977988-n8b75_6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7/console/0.log" Jun 03 16:24:18.953259 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.953088 2576 generic.go:358] "Generic (PLEG): container finished" podID="6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" containerID="803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b" exitCode=2 Jun 03 16:24:18.953259 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.953181 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9c977988-n8b75" event={"ID":"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7","Type":"ContainerDied","Data":"803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b"} Jun 03 16:24:18.953259 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.953218 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9c977988-n8b75" event={"ID":"6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7","Type":"ContainerDied","Data":"c758b6f360d2ecb96e6a9ad4c3a38eb05a3549c4d5b011a9df061c7d465b0343"} Jun 03 16:24:18.953259 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.953232 2576 scope.go:117] "RemoveContainer" containerID="803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b" Jun 03 16:24:18.953493 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.953195 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9c977988-n8b75" Jun 03 16:24:18.960976 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.960772 2576 scope.go:117] "RemoveContainer" containerID="803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b" Jun 03 16:24:18.961178 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:24:18.961039 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b\": container with ID starting with 803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b not found: ID does not exist" containerID="803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b" Jun 03 16:24:18.961178 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.961062 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b"} err="failed to get container status \"803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b\": rpc error: code = NotFound desc = could not find container \"803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b\": container with ID starting with 803f5339a0fe0201da094913431e268e9f8fdffcc217b0c6d8082372bbb1be1b not found: ID does not exist" Jun 03 16:24:18.973010 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.972987 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b9c977988-n8b75"] Jun 03 16:24:18.977640 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:18.977618 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b9c977988-n8b75"] Jun 03 16:24:20.312733 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:20.312699 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" path="/var/lib/kubelet/pods/6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7/volumes" Jun 03 16:24:26.746988 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.746955 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-djvpx"] Jun 03 16:24:26.747470 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.747248 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" containerName="console" Jun 03 16:24:26.747470 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.747260 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" containerName="console" Jun 03 16:24:26.747470 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.747334 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="6cbca0c6-3e3b-462e-8fc7-3ff14ab325e7" containerName="console" Jun 03 16:24:26.751681 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.751663 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.755177 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.755157 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 16:24:26.765768 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.765746 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-djvpx"] Jun 03 16:24:26.816397 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.816363 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ebd340a-d9e9-41b0-9fe9-b70951a03707-kubelet-config\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.816547 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.816443 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ebd340a-d9e9-41b0-9fe9-b70951a03707-dbus\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.816547 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.816480 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ebd340a-d9e9-41b0-9fe9-b70951a03707-original-pull-secret\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.917365 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.917295 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ebd340a-d9e9-41b0-9fe9-b70951a03707-dbus\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.917513 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.917391 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ebd340a-d9e9-41b0-9fe9-b70951a03707-original-pull-secret\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.917513 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.917458 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ebd340a-d9e9-41b0-9fe9-b70951a03707-kubelet-config\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.917575 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.917510 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ebd340a-d9e9-41b0-9fe9-b70951a03707-dbus\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.917575 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.917553 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ebd340a-d9e9-41b0-9fe9-b70951a03707-kubelet-config\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:26.919680 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:26.919660 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ebd340a-d9e9-41b0-9fe9-b70951a03707-original-pull-secret\") pod \"global-pull-secret-syncer-djvpx\" (UID: \"2ebd340a-d9e9-41b0-9fe9-b70951a03707\") " pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:27.060805 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:27.060773 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djvpx" Jun 03 16:24:27.194725 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:27.194701 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-djvpx"] Jun 03 16:24:27.197222 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:24:27.197180 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ebd340a_d9e9_41b0_9fe9_b70951a03707.slice/crio-854e256be4a485f62ac9545c1ca0785cd2ad593654f06645371cf3499c4dc1f3 WatchSource:0}: Error finding container 854e256be4a485f62ac9545c1ca0785cd2ad593654f06645371cf3499c4dc1f3: Status 404 returned error can't find the container with id 854e256be4a485f62ac9545c1ca0785cd2ad593654f06645371cf3499c4dc1f3 Jun 03 16:24:27.981723 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:27.981686 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-djvpx" event={"ID":"2ebd340a-d9e9-41b0-9fe9-b70951a03707","Type":"ContainerStarted","Data":"854e256be4a485f62ac9545c1ca0785cd2ad593654f06645371cf3499c4dc1f3"} Jun 03 16:24:28.991653 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:28.991567 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7ff79f989c-d4sh6" podUID="9940a591-cc0e-4561-8037-b890e648f56c" containerName="console" containerID="cri-o://7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6" gracePeriod=15 Jun 03 16:24:29.238765 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.238743 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7ff79f989c-d4sh6_9940a591-cc0e-4561-8037-b890e648f56c/console/0.log" Jun 03 16:24:29.238875 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.238808 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:24:29.337666 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.337638 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-oauth-serving-cert\") pod \"9940a591-cc0e-4561-8037-b890e648f56c\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " Jun 03 16:24:29.337832 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.337681 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-service-ca\") pod \"9940a591-cc0e-4561-8037-b890e648f56c\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " Jun 03 16:24:29.337832 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.337703 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-oauth-config\") pod \"9940a591-cc0e-4561-8037-b890e648f56c\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " Jun 03 16:24:29.337832 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.337726 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-console-config\") pod \"9940a591-cc0e-4561-8037-b890e648f56c\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " Jun 03 16:24:29.337832 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.337777 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fczrj\" (UniqueName: \"kubernetes.io/projected/9940a591-cc0e-4561-8037-b890e648f56c-kube-api-access-fczrj\") pod \"9940a591-cc0e-4561-8037-b890e648f56c\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " Jun 03 16:24:29.337832 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.337793 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-trusted-ca-bundle\") pod \"9940a591-cc0e-4561-8037-b890e648f56c\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " Jun 03 16:24:29.338085 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.337849 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-serving-cert\") pod \"9940a591-cc0e-4561-8037-b890e648f56c\" (UID: \"9940a591-cc0e-4561-8037-b890e648f56c\") " Jun 03 16:24:29.338085 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.338020 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9940a591-cc0e-4561-8037-b890e648f56c" (UID: "9940a591-cc0e-4561-8037-b890e648f56c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:24:29.338085 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.338067 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-service-ca" (OuterVolumeSpecName: "service-ca") pod "9940a591-cc0e-4561-8037-b890e648f56c" (UID: "9940a591-cc0e-4561-8037-b890e648f56c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:24:29.338239 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.338223 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9940a591-cc0e-4561-8037-b890e648f56c" (UID: "9940a591-cc0e-4561-8037-b890e648f56c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:24:29.338290 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.338221 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-console-config" (OuterVolumeSpecName: "console-config") pod "9940a591-cc0e-4561-8037-b890e648f56c" (UID: "9940a591-cc0e-4561-8037-b890e648f56c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:24:29.339804 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.339781 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9940a591-cc0e-4561-8037-b890e648f56c" (UID: "9940a591-cc0e-4561-8037-b890e648f56c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:24:29.339899 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.339884 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9940a591-cc0e-4561-8037-b890e648f56c" (UID: "9940a591-cc0e-4561-8037-b890e648f56c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:24:29.339934 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.339919 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9940a591-cc0e-4561-8037-b890e648f56c-kube-api-access-fczrj" (OuterVolumeSpecName: "kube-api-access-fczrj") pod "9940a591-cc0e-4561-8037-b890e648f56c" (UID: "9940a591-cc0e-4561-8037-b890e648f56c"). InnerVolumeSpecName "kube-api-access-fczrj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:24:29.439336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.439294 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-trusted-ca-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:29.439336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.439344 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:29.439557 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.439360 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-oauth-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:29.439557 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.439377 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-service-ca\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:29.439557 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.439392 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9940a591-cc0e-4561-8037-b890e648f56c-console-oauth-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:29.439557 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.439405 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9940a591-cc0e-4561-8037-b890e648f56c-console-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:29.439557 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.439419 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fczrj\" (UniqueName: \"kubernetes.io/projected/9940a591-cc0e-4561-8037-b890e648f56c-kube-api-access-fczrj\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:24:29.989631 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.989597 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7ff79f989c-d4sh6_9940a591-cc0e-4561-8037-b890e648f56c/console/0.log" Jun 03 16:24:29.989803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.989646 2576 generic.go:358] "Generic (PLEG): container finished" podID="9940a591-cc0e-4561-8037-b890e648f56c" containerID="7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6" exitCode=2 Jun 03 16:24:29.989803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.989715 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff79f989c-d4sh6" Jun 03 16:24:29.989803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.989708 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff79f989c-d4sh6" event={"ID":"9940a591-cc0e-4561-8037-b890e648f56c","Type":"ContainerDied","Data":"7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6"} Jun 03 16:24:29.989939 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.989832 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff79f989c-d4sh6" event={"ID":"9940a591-cc0e-4561-8037-b890e648f56c","Type":"ContainerDied","Data":"658b5ab335ed9d5e56cdb63eaf1eecdb6b9b4b840ca341bb28947e76ced6050b"} Jun 03 16:24:29.989939 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.989860 2576 scope.go:117] "RemoveContainer" containerID="7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6" Jun 03 16:24:29.999059 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.998964 2576 scope.go:117] "RemoveContainer" containerID="7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6" Jun 03 16:24:29.999357 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:24:29.999250 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6\": container with ID starting with 7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6 not found: ID does not exist" containerID="7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6" Jun 03 16:24:29.999357 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:29.999290 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6"} err="failed to get container status \"7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6\": rpc error: code = NotFound desc = could not find container \"7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6\": container with ID starting with 7c222f87cb076959ba647cc9d92c7b39267da268dc6b8fed0958878f6588d4f6 not found: ID does not exist" Jun 03 16:24:30.012884 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:30.012864 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ff79f989c-d4sh6"] Jun 03 16:24:30.016045 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:30.016019 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ff79f989c-d4sh6"] Jun 03 16:24:30.314576 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:30.314543 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9940a591-cc0e-4561-8037-b890e648f56c" path="/var/lib/kubelet/pods/9940a591-cc0e-4561-8037-b890e648f56c/volumes" Jun 03 16:24:31.997668 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:31.997582 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-djvpx" event={"ID":"2ebd340a-d9e9-41b0-9fe9-b70951a03707","Type":"ContainerStarted","Data":"53e6205a697c8dbe8d97cc410262da8202ea689d792a95456b2c2595fdca1be6"} Jun 03 16:24:32.016686 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:24:32.016624 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-djvpx" podStartSLOduration=1.701885367 podStartE2EDuration="6.016606799s" podCreationTimestamp="2026-06-03 16:24:26 +0000 UTC" firstStartedPulling="2026-06-03 16:24:27.200514093 +0000 UTC m=+197.464577951" lastFinishedPulling="2026-06-03 16:24:31.515235525 +0000 UTC m=+201.779299383" observedRunningTime="2026-06-03 16:24:32.01573262 +0000 UTC m=+202.279796497" watchObservedRunningTime="2026-06-03 16:24:32.016606799 +0000 UTC m=+202.280670675" Jun 03 16:25:01.475918 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.475881 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4"] Jun 03 16:25:01.476391 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.476190 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9940a591-cc0e-4561-8037-b890e648f56c" containerName="console" Jun 03 16:25:01.476391 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.476202 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9940a591-cc0e-4561-8037-b890e648f56c" containerName="console" Jun 03 16:25:01.476391 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.476248 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="9940a591-cc0e-4561-8037-b890e648f56c" containerName="console" Jun 03 16:25:01.478586 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.478571 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.481644 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.481619 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:25:01.481644 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.481622 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p5n4l\"" Jun 03 16:25:01.482426 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.482413 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:25:01.492282 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.492259 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwpch\" (UniqueName: \"kubernetes.io/projected/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-kube-api-access-hwpch\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.492397 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.492357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.492447 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.492410 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.494290 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.494268 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4"] Jun 03 16:25:01.592808 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.592770 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.593000 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.592816 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hwpch\" (UniqueName: \"kubernetes.io/projected/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-kube-api-access-hwpch\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.593000 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.592874 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.593209 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.593187 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.593209 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.593206 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.602745 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.602724 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwpch\" (UniqueName: \"kubernetes.io/projected/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-kube-api-access-hwpch\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.787768 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.787738 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:01.924505 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:01.924472 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4"] Jun 03 16:25:01.927741 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:25:01.927707 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedeeaf3c_7136_4a23_bc68_4fc4c283ea95.slice/crio-593616c09b10d7b6c0d1d19adfdff671cbc1d69ca5eaeb9376d6a5b200f2af40 WatchSource:0}: Error finding container 593616c09b10d7b6c0d1d19adfdff671cbc1d69ca5eaeb9376d6a5b200f2af40: Status 404 returned error can't find the container with id 593616c09b10d7b6c0d1d19adfdff671cbc1d69ca5eaeb9376d6a5b200f2af40 Jun 03 16:25:02.087044 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:02.086957 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" event={"ID":"edeeaf3c-7136-4a23-bc68-4fc4c283ea95","Type":"ContainerStarted","Data":"593616c09b10d7b6c0d1d19adfdff671cbc1d69ca5eaeb9376d6a5b200f2af40"} Jun 03 16:25:09.112230 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:09.112197 2576 generic.go:358] "Generic (PLEG): container finished" podID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerID="a727878b10741777a05d7a6227971d9c9592d13113e92233ceeca8b5c4f9f143" exitCode=0 Jun 03 16:25:09.112631 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:09.112272 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" event={"ID":"edeeaf3c-7136-4a23-bc68-4fc4c283ea95","Type":"ContainerDied","Data":"a727878b10741777a05d7a6227971d9c9592d13113e92233ceeca8b5c4f9f143"} Jun 03 16:25:12.122443 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:12.122409 2576 generic.go:358] "Generic (PLEG): container finished" podID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerID="45bd7ead14e5af718412b73b37bd4d0d45a317d5493ff2c60bead8a4237f04ef" exitCode=0 Jun 03 16:25:12.122823 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:12.122485 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" event={"ID":"edeeaf3c-7136-4a23-bc68-4fc4c283ea95","Type":"ContainerDied","Data":"45bd7ead14e5af718412b73b37bd4d0d45a317d5493ff2c60bead8a4237f04ef"} Jun 03 16:25:20.146427 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:20.146342 2576 generic.go:358] "Generic (PLEG): container finished" podID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerID="ee2cd479a66961c9dd3f3311cb208cce045cd1a49220c62fc59754d0e48a3579" exitCode=0 Jun 03 16:25:20.146787 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:20.146424 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" event={"ID":"edeeaf3c-7136-4a23-bc68-4fc4c283ea95","Type":"ContainerDied","Data":"ee2cd479a66961c9dd3f3311cb208cce045cd1a49220c62fc59754d0e48a3579"} Jun 03 16:25:21.270704 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.270683 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:21.360305 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.360267 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-bundle\") pod \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " Jun 03 16:25:21.360512 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.360341 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-util\") pod \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " Jun 03 16:25:21.360512 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.360362 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwpch\" (UniqueName: \"kubernetes.io/projected/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-kube-api-access-hwpch\") pod \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\" (UID: \"edeeaf3c-7136-4a23-bc68-4fc4c283ea95\") " Jun 03 16:25:21.360996 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.360969 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-bundle" (OuterVolumeSpecName: "bundle") pod "edeeaf3c-7136-4a23-bc68-4fc4c283ea95" (UID: "edeeaf3c-7136-4a23-bc68-4fc4c283ea95"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:21.362582 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.362562 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-kube-api-access-hwpch" (OuterVolumeSpecName: "kube-api-access-hwpch") pod "edeeaf3c-7136-4a23-bc68-4fc4c283ea95" (UID: "edeeaf3c-7136-4a23-bc68-4fc4c283ea95"). InnerVolumeSpecName "kube-api-access-hwpch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:21.364480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.364461 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-util" (OuterVolumeSpecName: "util") pod "edeeaf3c-7136-4a23-bc68-4fc4c283ea95" (UID: "edeeaf3c-7136-4a23-bc68-4fc4c283ea95"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:21.461733 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.461648 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:25:21.461733 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.461677 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwpch\" (UniqueName: \"kubernetes.io/projected/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-kube-api-access-hwpch\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:25:21.461733 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:21.461688 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edeeaf3c-7136-4a23-bc68-4fc4c283ea95-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:25:22.154335 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:22.154290 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" Jun 03 16:25:22.154335 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:22.154289 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5psw4" event={"ID":"edeeaf3c-7136-4a23-bc68-4fc4c283ea95","Type":"ContainerDied","Data":"593616c09b10d7b6c0d1d19adfdff671cbc1d69ca5eaeb9376d6a5b200f2af40"} Jun 03 16:25:22.154530 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:22.154353 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="593616c09b10d7b6c0d1d19adfdff671cbc1d69ca5eaeb9376d6a5b200f2af40" Jun 03 16:25:29.726857 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.726818 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj"] Jun 03 16:25:29.727222 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.727105 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerName="extract" Jun 03 16:25:29.727222 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.727115 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerName="extract" Jun 03 16:25:29.727222 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.727126 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerName="pull" Jun 03 16:25:29.727222 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.727131 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerName="pull" Jun 03 16:25:29.727222 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.727143 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerName="util" Jun 03 16:25:29.727222 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.727148 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerName="util" Jun 03 16:25:29.727222 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.727191 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="edeeaf3c-7136-4a23-bc68-4fc4c283ea95" containerName="extract" Jun 03 16:25:29.731382 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.731366 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" Jun 03 16:25:29.734509 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.734488 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 03 16:25:29.734883 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.734859 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:25:29.735449 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.735436 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-pjh5b\"" Jun 03 16:25:29.761027 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.761002 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj"] Jun 03 16:25:29.825257 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.825224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2rdv\" (UniqueName: \"kubernetes.io/projected/4fbe0ae7-2e98-4a09-87cb-5477a4041a31-kube-api-access-q2rdv\") pod \"cert-manager-operator-controller-manager-54b9655956-cb8fj\" (UID: \"4fbe0ae7-2e98-4a09-87cb-5477a4041a31\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" Jun 03 16:25:29.825416 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.825263 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fbe0ae7-2e98-4a09-87cb-5477a4041a31-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-cb8fj\" (UID: \"4fbe0ae7-2e98-4a09-87cb-5477a4041a31\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" Jun 03 16:25:29.925906 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.925860 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q2rdv\" (UniqueName: \"kubernetes.io/projected/4fbe0ae7-2e98-4a09-87cb-5477a4041a31-kube-api-access-q2rdv\") pod \"cert-manager-operator-controller-manager-54b9655956-cb8fj\" (UID: \"4fbe0ae7-2e98-4a09-87cb-5477a4041a31\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" Jun 03 16:25:29.925906 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.925908 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fbe0ae7-2e98-4a09-87cb-5477a4041a31-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-cb8fj\" (UID: \"4fbe0ae7-2e98-4a09-87cb-5477a4041a31\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" Jun 03 16:25:29.926217 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.926202 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fbe0ae7-2e98-4a09-87cb-5477a4041a31-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-cb8fj\" (UID: \"4fbe0ae7-2e98-4a09-87cb-5477a4041a31\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" Jun 03 16:25:29.948483 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:29.948455 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2rdv\" (UniqueName: \"kubernetes.io/projected/4fbe0ae7-2e98-4a09-87cb-5477a4041a31-kube-api-access-q2rdv\") pod \"cert-manager-operator-controller-manager-54b9655956-cb8fj\" (UID: \"4fbe0ae7-2e98-4a09-87cb-5477a4041a31\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" Jun 03 16:25:30.040278 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:30.040243 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" Jun 03 16:25:30.203309 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:30.203259 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj"] Jun 03 16:25:30.208418 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:25:30.208392 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbe0ae7_2e98_4a09_87cb_5477a4041a31.slice/crio-0d65330a4f7769a7636dab617bb7d997b932212187b666127b05c53b5ae9d56f WatchSource:0}: Error finding container 0d65330a4f7769a7636dab617bb7d997b932212187b666127b05c53b5ae9d56f: Status 404 returned error can't find the container with id 0d65330a4f7769a7636dab617bb7d997b932212187b666127b05c53b5ae9d56f Jun 03 16:25:31.180059 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:31.180011 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" event={"ID":"4fbe0ae7-2e98-4a09-87cb-5477a4041a31","Type":"ContainerStarted","Data":"0d65330a4f7769a7636dab617bb7d997b932212187b666127b05c53b5ae9d56f"} Jun 03 16:25:32.184852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:32.184814 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" event={"ID":"4fbe0ae7-2e98-4a09-87cb-5477a4041a31","Type":"ContainerStarted","Data":"284abe814bfebd8bc6ce38e9cce336a3cca46805d600289ec5a65949bb11369a"} Jun 03 16:25:32.228908 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:32.228846 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-cb8fj" podStartSLOduration=1.754082406 podStartE2EDuration="3.228826818s" podCreationTimestamp="2026-06-03 16:25:29 +0000 UTC" firstStartedPulling="2026-06-03 16:25:30.210810035 +0000 UTC m=+260.474873888" lastFinishedPulling="2026-06-03 16:25:31.685554444 +0000 UTC m=+261.949618300" observedRunningTime="2026-06-03 16:25:32.225484584 +0000 UTC m=+262.489548458" watchObservedRunningTime="2026-06-03 16:25:32.228826818 +0000 UTC m=+262.492890695" Jun 03 16:25:33.683549 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.683519 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg"] Jun 03 16:25:33.686262 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.686244 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.689641 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.689625 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p5n4l\"" Jun 03 16:25:33.690074 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.690061 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:25:33.690175 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.690160 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:25:33.705481 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.705459 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg"] Jun 03 16:25:33.763283 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.763251 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5phd8\" (UniqueName: \"kubernetes.io/projected/ba947699-d31d-4073-90ac-6be148f752b6-kube-api-access-5phd8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.763442 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.763291 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.763442 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.763398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.780760 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.780725 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-mhfbr"] Jun 03 16:25:33.782976 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.782957 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:33.785588 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.785570 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 03 16:25:33.785855 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.785839 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 03 16:25:33.785935 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.785908 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-zzx4q\"" Jun 03 16:25:33.792942 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.792923 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-mhfbr"] Jun 03 16:25:33.864344 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.864287 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.864507 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.864354 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5phd8\" (UniqueName: \"kubernetes.io/projected/ba947699-d31d-4073-90ac-6be148f752b6-kube-api-access-5phd8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.864507 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.864382 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.864507 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.864406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1861e07-91a2-42dc-aaf2-47f686ea3c97-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-mhfbr\" (UID: \"b1861e07-91a2-42dc-aaf2-47f686ea3c97\") " pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:33.864507 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.864436 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2rhc\" (UniqueName: \"kubernetes.io/projected/b1861e07-91a2-42dc-aaf2-47f686ea3c97-kube-api-access-l2rhc\") pod \"cert-manager-webhook-587ccfb98-mhfbr\" (UID: \"b1861e07-91a2-42dc-aaf2-47f686ea3c97\") " pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:33.864744 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.864727 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.864797 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.864783 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.873216 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.873188 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5phd8\" (UniqueName: \"kubernetes.io/projected/ba947699-d31d-4073-90ac-6be148f752b6-kube-api-access-5phd8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:33.965776 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.965681 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1861e07-91a2-42dc-aaf2-47f686ea3c97-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-mhfbr\" (UID: \"b1861e07-91a2-42dc-aaf2-47f686ea3c97\") " pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:33.965776 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.965740 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l2rhc\" (UniqueName: \"kubernetes.io/projected/b1861e07-91a2-42dc-aaf2-47f686ea3c97-kube-api-access-l2rhc\") pod \"cert-manager-webhook-587ccfb98-mhfbr\" (UID: \"b1861e07-91a2-42dc-aaf2-47f686ea3c97\") " pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:33.977857 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.977828 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2rhc\" (UniqueName: \"kubernetes.io/projected/b1861e07-91a2-42dc-aaf2-47f686ea3c97-kube-api-access-l2rhc\") pod \"cert-manager-webhook-587ccfb98-mhfbr\" (UID: \"b1861e07-91a2-42dc-aaf2-47f686ea3c97\") " pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:33.977952 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.977931 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1861e07-91a2-42dc-aaf2-47f686ea3c97-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-mhfbr\" (UID: \"b1861e07-91a2-42dc-aaf2-47f686ea3c97\") " pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:33.994868 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:33.994849 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:34.101630 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:34.101603 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:34.123866 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:34.123830 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg"] Jun 03 16:25:34.124778 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:25:34.124741 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba947699_d31d_4073_90ac_6be148f752b6.slice/crio-615a279d53cd621f08ce87434bea0ae97cb238449c22106fc16a3feb24138e51 WatchSource:0}: Error finding container 615a279d53cd621f08ce87434bea0ae97cb238449c22106fc16a3feb24138e51: Status 404 returned error can't find the container with id 615a279d53cd621f08ce87434bea0ae97cb238449c22106fc16a3feb24138e51 Jun 03 16:25:34.196099 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:34.196040 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" event={"ID":"ba947699-d31d-4073-90ac-6be148f752b6","Type":"ContainerStarted","Data":"fe32c809e0edb3db53521ed67b7170d84f38ecc4b28ee284782eeaa3a1fa2802"} Jun 03 16:25:34.196099 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:34.196079 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" event={"ID":"ba947699-d31d-4073-90ac-6be148f752b6","Type":"ContainerStarted","Data":"615a279d53cd621f08ce87434bea0ae97cb238449c22106fc16a3feb24138e51"} Jun 03 16:25:34.235613 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:34.235577 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-mhfbr"] Jun 03 16:25:34.246087 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:25:34.246062 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1861e07_91a2_42dc_aaf2_47f686ea3c97.slice/crio-389845f9660ae798c69ff39380e591f281ad3907d1b742307f76ede4da0ca5d9 WatchSource:0}: Error finding container 389845f9660ae798c69ff39380e591f281ad3907d1b742307f76ede4da0ca5d9: Status 404 returned error can't find the container with id 389845f9660ae798c69ff39380e591f281ad3907d1b742307f76ede4da0ca5d9 Jun 03 16:25:35.201191 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:35.201154 2576 generic.go:358] "Generic (PLEG): container finished" podID="ba947699-d31d-4073-90ac-6be148f752b6" containerID="fe32c809e0edb3db53521ed67b7170d84f38ecc4b28ee284782eeaa3a1fa2802" exitCode=0 Jun 03 16:25:35.201658 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:35.201244 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" event={"ID":"ba947699-d31d-4073-90ac-6be148f752b6","Type":"ContainerDied","Data":"fe32c809e0edb3db53521ed67b7170d84f38ecc4b28ee284782eeaa3a1fa2802"} Jun 03 16:25:35.202438 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:35.202416 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" event={"ID":"b1861e07-91a2-42dc-aaf2-47f686ea3c97","Type":"ContainerStarted","Data":"389845f9660ae798c69ff39380e591f281ad3907d1b742307f76ede4da0ca5d9"} Jun 03 16:25:37.870121 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:37.870096 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-9j2dv"] Jun 03 16:25:37.872485 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:37.872467 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" Jun 03 16:25:37.874828 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:37.874812 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-66qcm\"" Jun 03 16:25:37.883240 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:37.883218 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-9j2dv"] Jun 03 16:25:38.002370 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.002345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwzgr\" (UniqueName: \"kubernetes.io/projected/21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f-kube-api-access-wwzgr\") pod \"cert-manager-cainjector-68b757865b-9j2dv\" (UID: \"21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f\") " pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" Jun 03 16:25:38.002488 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.002379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-9j2dv\" (UID: \"21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f\") " pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" Jun 03 16:25:38.103647 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.103622 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wwzgr\" (UniqueName: \"kubernetes.io/projected/21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f-kube-api-access-wwzgr\") pod \"cert-manager-cainjector-68b757865b-9j2dv\" (UID: \"21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f\") " pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" Jun 03 16:25:38.103747 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.103653 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-9j2dv\" (UID: \"21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f\") " pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" Jun 03 16:25:38.117564 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.117537 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-9j2dv\" (UID: \"21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f\") " pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" Jun 03 16:25:38.117691 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.117676 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwzgr\" (UniqueName: \"kubernetes.io/projected/21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f-kube-api-access-wwzgr\") pod \"cert-manager-cainjector-68b757865b-9j2dv\" (UID: \"21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f\") " pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" Jun 03 16:25:38.181245 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.181168 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" Jun 03 16:25:38.213502 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.213470 2576 generic.go:358] "Generic (PLEG): container finished" podID="ba947699-d31d-4073-90ac-6be148f752b6" containerID="6e7a2b296cce3f7fea7481cfdf809e0d310f8cafc71df5f5a1aa6767a0b415fa" exitCode=0 Jun 03 16:25:38.213647 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.213549 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" event={"ID":"ba947699-d31d-4073-90ac-6be148f752b6","Type":"ContainerDied","Data":"6e7a2b296cce3f7fea7481cfdf809e0d310f8cafc71df5f5a1aa6767a0b415fa"} Jun 03 16:25:38.214874 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.214843 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" event={"ID":"b1861e07-91a2-42dc-aaf2-47f686ea3c97","Type":"ContainerStarted","Data":"7279a6c6822660456256d50979d9fe084a565a6144397a37e7ced2ebca4bf9b9"} Jun 03 16:25:38.214995 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.214978 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:38.254554 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.254505 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" podStartSLOduration=2.01901628 podStartE2EDuration="5.254492082s" podCreationTimestamp="2026-06-03 16:25:33 +0000 UTC" firstStartedPulling="2026-06-03 16:25:34.247770015 +0000 UTC m=+264.511833868" lastFinishedPulling="2026-06-03 16:25:37.483245808 +0000 UTC m=+267.747309670" observedRunningTime="2026-06-03 16:25:38.251795269 +0000 UTC m=+268.515859144" watchObservedRunningTime="2026-06-03 16:25:38.254492082 +0000 UTC m=+268.518555957" Jun 03 16:25:38.319590 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:38.319567 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-9j2dv"] Jun 03 16:25:38.320881 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:25:38.320861 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21fd3014_b634_4e43_9e3d_9b0c0d6e9e6f.slice/crio-de629dbbb2645622208d2b6fdb9bfaf5d2fa45add5b86305b878c0b5c4725bab WatchSource:0}: Error finding container de629dbbb2645622208d2b6fdb9bfaf5d2fa45add5b86305b878c0b5c4725bab: Status 404 returned error can't find the container with id de629dbbb2645622208d2b6fdb9bfaf5d2fa45add5b86305b878c0b5c4725bab Jun 03 16:25:39.220564 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:39.220528 2576 generic.go:358] "Generic (PLEG): container finished" podID="ba947699-d31d-4073-90ac-6be148f752b6" containerID="e125834eef07f7ae1ef59c40040761c12626da476b7646b22534f5f0597daa28" exitCode=0 Jun 03 16:25:39.221014 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:39.220608 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" event={"ID":"ba947699-d31d-4073-90ac-6be148f752b6","Type":"ContainerDied","Data":"e125834eef07f7ae1ef59c40040761c12626da476b7646b22534f5f0597daa28"} Jun 03 16:25:39.221883 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:39.221854 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" event={"ID":"21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f","Type":"ContainerStarted","Data":"5732344d46720e911a01b5b2b1fa44a7acc0301ff5959020cd254dc70947e01a"} Jun 03 16:25:39.222018 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:39.221889 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" event={"ID":"21fd3014-b634-4e43-9e3d-9b0c0d6e9e6f","Type":"ContainerStarted","Data":"de629dbbb2645622208d2b6fdb9bfaf5d2fa45add5b86305b878c0b5c4725bab"} Jun 03 16:25:39.260070 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:39.260027 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-9j2dv" podStartSLOduration=2.260011324 podStartE2EDuration="2.260011324s" podCreationTimestamp="2026-06-03 16:25:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:25:39.25913629 +0000 UTC m=+269.523200166" watchObservedRunningTime="2026-06-03 16:25:39.260011324 +0000 UTC m=+269.524075201" Jun 03 16:25:40.352865 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.352843 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:40.523871 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.523838 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-util\") pod \"ba947699-d31d-4073-90ac-6be148f752b6\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " Jun 03 16:25:40.524038 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.523892 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5phd8\" (UniqueName: \"kubernetes.io/projected/ba947699-d31d-4073-90ac-6be148f752b6-kube-api-access-5phd8\") pod \"ba947699-d31d-4073-90ac-6be148f752b6\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " Jun 03 16:25:40.524038 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.523941 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-bundle\") pod \"ba947699-d31d-4073-90ac-6be148f752b6\" (UID: \"ba947699-d31d-4073-90ac-6be148f752b6\") " Jun 03 16:25:40.524333 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.524300 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-bundle" (OuterVolumeSpecName: "bundle") pod "ba947699-d31d-4073-90ac-6be148f752b6" (UID: "ba947699-d31d-4073-90ac-6be148f752b6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:40.525854 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.525819 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba947699-d31d-4073-90ac-6be148f752b6-kube-api-access-5phd8" (OuterVolumeSpecName: "kube-api-access-5phd8") pod "ba947699-d31d-4073-90ac-6be148f752b6" (UID: "ba947699-d31d-4073-90ac-6be148f752b6"). InnerVolumeSpecName "kube-api-access-5phd8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:40.530089 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.530062 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-util" (OuterVolumeSpecName: "util") pod "ba947699-d31d-4073-90ac-6be148f752b6" (UID: "ba947699-d31d-4073-90ac-6be148f752b6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:40.625168 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.625143 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:25:40.625168 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.625164 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba947699-d31d-4073-90ac-6be148f752b6-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:25:40.625292 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:40.625174 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5phd8\" (UniqueName: \"kubernetes.io/projected/ba947699-d31d-4073-90ac-6be148f752b6-kube-api-access-5phd8\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:25:41.229973 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:41.229940 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" event={"ID":"ba947699-d31d-4073-90ac-6be148f752b6","Type":"ContainerDied","Data":"615a279d53cd621f08ce87434bea0ae97cb238449c22106fc16a3feb24138e51"} Jun 03 16:25:41.229973 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:41.229974 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="615a279d53cd621f08ce87434bea0ae97cb238449c22106fc16a3feb24138e51" Jun 03 16:25:41.230205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:41.230005 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm7vtg" Jun 03 16:25:44.223961 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:44.223933 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-mhfbr" Jun 03 16:25:45.238612 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.238580 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-gxwk2"] Jun 03 16:25:45.238990 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.238884 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba947699-d31d-4073-90ac-6be148f752b6" containerName="util" Jun 03 16:25:45.238990 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.238898 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba947699-d31d-4073-90ac-6be148f752b6" containerName="util" Jun 03 16:25:45.238990 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.238911 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba947699-d31d-4073-90ac-6be148f752b6" containerName="pull" Jun 03 16:25:45.238990 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.238916 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba947699-d31d-4073-90ac-6be148f752b6" containerName="pull" Jun 03 16:25:45.238990 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.238925 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba947699-d31d-4073-90ac-6be148f752b6" containerName="extract" Jun 03 16:25:45.238990 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.238931 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba947699-d31d-4073-90ac-6be148f752b6" containerName="extract" Jun 03 16:25:45.239213 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.238996 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ba947699-d31d-4073-90ac-6be148f752b6" containerName="extract" Jun 03 16:25:45.242655 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.242639 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-gxwk2" Jun 03 16:25:45.246402 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.246382 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hkjm5\"" Jun 03 16:25:45.258175 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.258154 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-gxwk2"] Jun 03 16:25:45.362974 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.362945 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a62ee868-5562-41bf-8261-2540e1a904e4-bound-sa-token\") pod \"cert-manager-79c8d999ff-gxwk2\" (UID: \"a62ee868-5562-41bf-8261-2540e1a904e4\") " pod="cert-manager/cert-manager-79c8d999ff-gxwk2" Jun 03 16:25:45.363119 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.362995 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4c6k\" (UniqueName: \"kubernetes.io/projected/a62ee868-5562-41bf-8261-2540e1a904e4-kube-api-access-c4c6k\") pod \"cert-manager-79c8d999ff-gxwk2\" (UID: \"a62ee868-5562-41bf-8261-2540e1a904e4\") " pod="cert-manager/cert-manager-79c8d999ff-gxwk2" Jun 03 16:25:45.463911 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.463863 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a62ee868-5562-41bf-8261-2540e1a904e4-bound-sa-token\") pod \"cert-manager-79c8d999ff-gxwk2\" (UID: \"a62ee868-5562-41bf-8261-2540e1a904e4\") " pod="cert-manager/cert-manager-79c8d999ff-gxwk2" Jun 03 16:25:45.464109 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.463962 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4c6k\" (UniqueName: \"kubernetes.io/projected/a62ee868-5562-41bf-8261-2540e1a904e4-kube-api-access-c4c6k\") pod \"cert-manager-79c8d999ff-gxwk2\" (UID: \"a62ee868-5562-41bf-8261-2540e1a904e4\") " pod="cert-manager/cert-manager-79c8d999ff-gxwk2" Jun 03 16:25:45.473230 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.473195 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a62ee868-5562-41bf-8261-2540e1a904e4-bound-sa-token\") pod \"cert-manager-79c8d999ff-gxwk2\" (UID: \"a62ee868-5562-41bf-8261-2540e1a904e4\") " pod="cert-manager/cert-manager-79c8d999ff-gxwk2" Jun 03 16:25:45.473364 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.473276 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4c6k\" (UniqueName: \"kubernetes.io/projected/a62ee868-5562-41bf-8261-2540e1a904e4-kube-api-access-c4c6k\") pod \"cert-manager-79c8d999ff-gxwk2\" (UID: \"a62ee868-5562-41bf-8261-2540e1a904e4\") " pod="cert-manager/cert-manager-79c8d999ff-gxwk2" Jun 03 16:25:45.551036 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.550987 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-gxwk2" Jun 03 16:25:45.670753 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:45.670717 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-gxwk2"] Jun 03 16:25:45.673628 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:25:45.673597 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda62ee868_5562_41bf_8261_2540e1a904e4.slice/crio-3510e3a913a6e94d2a04855c970c6e442bdc74b7442d7fee74b0367645816adc WatchSource:0}: Error finding container 3510e3a913a6e94d2a04855c970c6e442bdc74b7442d7fee74b0367645816adc: Status 404 returned error can't find the container with id 3510e3a913a6e94d2a04855c970c6e442bdc74b7442d7fee74b0367645816adc Jun 03 16:25:46.247102 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:46.247069 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-gxwk2" event={"ID":"a62ee868-5562-41bf-8261-2540e1a904e4","Type":"ContainerStarted","Data":"867d69631f0a2381a7b6bea7301d67a8984ade16a9817558f71c3ea9a255f024"} Jun 03 16:25:46.247102 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:46.247101 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-gxwk2" event={"ID":"a62ee868-5562-41bf-8261-2540e1a904e4","Type":"ContainerStarted","Data":"3510e3a913a6e94d2a04855c970c6e442bdc74b7442d7fee74b0367645816adc"} Jun 03 16:25:46.266342 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:46.266275 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-gxwk2" podStartSLOduration=1.266260941 podStartE2EDuration="1.266260941s" podCreationTimestamp="2026-06-03 16:25:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:25:46.265032974 +0000 UTC m=+276.529096846" watchObservedRunningTime="2026-06-03 16:25:46.266260941 +0000 UTC m=+276.530324815" Jun 03 16:25:57.594894 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.594865 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl"] Jun 03 16:25:57.599987 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.599964 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.601850 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.601823 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl"] Jun 03 16:25:57.602524 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.602502 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:25:57.603480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.603462 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:25:57.603566 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.603465 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p5n4l\"" Jun 03 16:25:57.650384 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.650359 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.650468 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.650391 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9xxf\" (UniqueName: \"kubernetes.io/projected/30dae02d-c6c3-4bea-b667-3ec404bb85a3-kube-api-access-c9xxf\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.650468 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.650418 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.751417 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.751394 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.751520 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.751451 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.751582 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.751559 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c9xxf\" (UniqueName: \"kubernetes.io/projected/30dae02d-c6c3-4bea-b667-3ec404bb85a3-kube-api-access-c9xxf\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.751777 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.751762 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.751860 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.751842 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.760814 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.760785 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9xxf\" (UniqueName: \"kubernetes.io/projected/30dae02d-c6c3-4bea-b667-3ec404bb85a3-kube-api-access-c9xxf\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:57.910188 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:57.910139 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:25:58.034176 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:58.034152 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl"] Jun 03 16:25:58.036719 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:25:58.036691 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30dae02d_c6c3_4bea_b667_3ec404bb85a3.slice/crio-7928190259b87088b049ce0a11072f961fa0649b1b0401f9d76828aa2742f214 WatchSource:0}: Error finding container 7928190259b87088b049ce0a11072f961fa0649b1b0401f9d76828aa2742f214: Status 404 returned error can't find the container with id 7928190259b87088b049ce0a11072f961fa0649b1b0401f9d76828aa2742f214 Jun 03 16:25:58.290402 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:58.290371 2576 generic.go:358] "Generic (PLEG): container finished" podID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerID="1d8eafec1819be1867c7dad0b67f6a2daecc53ff0ebe17cdf68b77c3d6b9b924" exitCode=0 Jun 03 16:25:58.290515 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:58.290445 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" event={"ID":"30dae02d-c6c3-4bea-b667-3ec404bb85a3","Type":"ContainerDied","Data":"1d8eafec1819be1867c7dad0b67f6a2daecc53ff0ebe17cdf68b77c3d6b9b924"} Jun 03 16:25:58.290515 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:58.290474 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" event={"ID":"30dae02d-c6c3-4bea-b667-3ec404bb85a3","Type":"ContainerStarted","Data":"7928190259b87088b049ce0a11072f961fa0649b1b0401f9d76828aa2742f214"} Jun 03 16:25:59.295698 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:59.295665 2576 generic.go:358] "Generic (PLEG): container finished" podID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerID="6912bf611d9b1f2a4845e02883684da661821003a00387eef316428bf364f46d" exitCode=0 Jun 03 16:25:59.296052 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:25:59.295705 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" event={"ID":"30dae02d-c6c3-4bea-b667-3ec404bb85a3","Type":"ContainerDied","Data":"6912bf611d9b1f2a4845e02883684da661821003a00387eef316428bf364f46d"} Jun 03 16:26:00.300929 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:00.300897 2576 generic.go:358] "Generic (PLEG): container finished" podID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerID="298c56ace813e1daf29761b33b23e9172086a656be5a7527b164f7dc81cb6be7" exitCode=0 Jun 03 16:26:00.301281 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:00.300989 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" event={"ID":"30dae02d-c6c3-4bea-b667-3ec404bb85a3","Type":"ContainerDied","Data":"298c56ace813e1daf29761b33b23e9172086a656be5a7527b164f7dc81cb6be7"} Jun 03 16:26:01.425678 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.425659 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:26:01.475149 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.475122 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-util\") pod \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " Jun 03 16:26:01.475293 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.475201 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-bundle\") pod \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " Jun 03 16:26:01.475293 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.475221 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9xxf\" (UniqueName: \"kubernetes.io/projected/30dae02d-c6c3-4bea-b667-3ec404bb85a3-kube-api-access-c9xxf\") pod \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\" (UID: \"30dae02d-c6c3-4bea-b667-3ec404bb85a3\") " Jun 03 16:26:01.475922 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.475895 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-bundle" (OuterVolumeSpecName: "bundle") pod "30dae02d-c6c3-4bea-b667-3ec404bb85a3" (UID: "30dae02d-c6c3-4bea-b667-3ec404bb85a3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:26:01.477351 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.477309 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30dae02d-c6c3-4bea-b667-3ec404bb85a3-kube-api-access-c9xxf" (OuterVolumeSpecName: "kube-api-access-c9xxf") pod "30dae02d-c6c3-4bea-b667-3ec404bb85a3" (UID: "30dae02d-c6c3-4bea-b667-3ec404bb85a3"). InnerVolumeSpecName "kube-api-access-c9xxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:26:01.480712 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.480689 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-util" (OuterVolumeSpecName: "util") pod "30dae02d-c6c3-4bea-b667-3ec404bb85a3" (UID: "30dae02d-c6c3-4bea-b667-3ec404bb85a3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:26:01.575859 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.575802 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:01.575859 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.575825 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9xxf\" (UniqueName: \"kubernetes.io/projected/30dae02d-c6c3-4bea-b667-3ec404bb85a3-kube-api-access-c9xxf\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:01.575859 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:01.575835 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30dae02d-c6c3-4bea-b667-3ec404bb85a3-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:02.309152 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:02.309126 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" Jun 03 16:26:02.312644 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:02.312619 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdb7hgl" event={"ID":"30dae02d-c6c3-4bea-b667-3ec404bb85a3","Type":"ContainerDied","Data":"7928190259b87088b049ce0a11072f961fa0649b1b0401f9d76828aa2742f214"} Jun 03 16:26:02.312644 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:02.312645 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7928190259b87088b049ce0a11072f961fa0649b1b0401f9d76828aa2742f214" Jun 03 16:26:10.195544 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:10.195518 2576 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 16:26:12.566945 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.566914 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx"] Jun 03 16:26:12.568414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.567335 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerName="pull" Jun 03 16:26:12.568414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.567351 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerName="pull" Jun 03 16:26:12.568414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.567369 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerName="util" Jun 03 16:26:12.568414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.567375 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerName="util" Jun 03 16:26:12.568414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.567385 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerName="extract" Jun 03 16:26:12.568414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.567390 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerName="extract" Jun 03 16:26:12.568414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.567447 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="30dae02d-c6c3-4bea-b667-3ec404bb85a3" containerName="extract" Jun 03 16:26:12.569265 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.569247 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.572054 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.572019 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 03 16:26:12.572212 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.572092 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 03 16:26:12.572212 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.572104 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bbdcs\"" Jun 03 16:26:12.573502 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.573487 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 03 16:26:12.573893 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.573879 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 03 16:26:12.585766 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.585739 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx"] Jun 03 16:26:12.659082 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.659056 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/35147149-64fa-43cc-b747-cc40bbac566a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.659194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.659088 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmrjz\" (UniqueName: \"kubernetes.io/projected/35147149-64fa-43cc-b747-cc40bbac566a-kube-api-access-pmrjz\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.659194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.659106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35147149-64fa-43cc-b747-cc40bbac566a-webhook-cert\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.659194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.659121 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/35147149-64fa-43cc-b747-cc40bbac566a-manifests\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.659194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.659140 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35147149-64fa-43cc-b747-cc40bbac566a-tmp\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.759702 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.759669 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/35147149-64fa-43cc-b747-cc40bbac566a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.759837 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.759708 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pmrjz\" (UniqueName: \"kubernetes.io/projected/35147149-64fa-43cc-b747-cc40bbac566a-kube-api-access-pmrjz\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.759837 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.759725 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35147149-64fa-43cc-b747-cc40bbac566a-webhook-cert\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.759837 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.759740 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/35147149-64fa-43cc-b747-cc40bbac566a-manifests\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.759837 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.759806 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35147149-64fa-43cc-b747-cc40bbac566a-tmp\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.760194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.760171 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/35147149-64fa-43cc-b747-cc40bbac566a-manifests\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.762048 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.762019 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35147149-64fa-43cc-b747-cc40bbac566a-tmp\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.762153 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.762087 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35147149-64fa-43cc-b747-cc40bbac566a-webhook-cert\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.762231 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.762213 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/35147149-64fa-43cc-b747-cc40bbac566a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.770669 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.770645 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmrjz\" (UniqueName: \"kubernetes.io/projected/35147149-64fa-43cc-b747-cc40bbac566a-kube-api-access-pmrjz\") pod \"opendatahub-operator-controller-manager-84984b75f8-dnlsx\" (UID: \"35147149-64fa-43cc-b747-cc40bbac566a\") " pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:12.879480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:12.879419 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:13.010475 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:13.010443 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx"] Jun 03 16:26:13.016606 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:13.016578 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35147149_64fa_43cc_b747_cc40bbac566a.slice/crio-623389b1749f8da891d2edd70bd2756fe02629156ccf6a15dd8a9dcd4b95a562 WatchSource:0}: Error finding container 623389b1749f8da891d2edd70bd2756fe02629156ccf6a15dd8a9dcd4b95a562: Status 404 returned error can't find the container with id 623389b1749f8da891d2edd70bd2756fe02629156ccf6a15dd8a9dcd4b95a562 Jun 03 16:26:13.018289 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:13.018274 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:26:13.346790 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:13.346761 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" event={"ID":"35147149-64fa-43cc-b747-cc40bbac566a","Type":"ContainerStarted","Data":"623389b1749f8da891d2edd70bd2756fe02629156ccf6a15dd8a9dcd4b95a562"} Jun 03 16:26:16.357215 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:16.357181 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" event={"ID":"35147149-64fa-43cc-b747-cc40bbac566a","Type":"ContainerStarted","Data":"8a19827c6132238d6a5cd1820645529c3536323d489b59295e0acb5820255e3b"} Jun 03 16:26:17.361982 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:17.361951 2576 generic.go:358] "Generic (PLEG): container finished" podID="35147149-64fa-43cc-b747-cc40bbac566a" containerID="8a19827c6132238d6a5cd1820645529c3536323d489b59295e0acb5820255e3b" exitCode=0 Jun 03 16:26:17.362337 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:17.362034 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" event={"ID":"35147149-64fa-43cc-b747-cc40bbac566a","Type":"ContainerDied","Data":"8a19827c6132238d6a5cd1820645529c3536323d489b59295e0acb5820255e3b"} Jun 03 16:26:18.366830 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:18.366792 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" event={"ID":"35147149-64fa-43cc-b747-cc40bbac566a","Type":"ContainerStarted","Data":"93510441a067c9097b52f55b703ce5cac4d298084c2c484d792ea5e7086a7839"} Jun 03 16:26:18.367205 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:18.366899 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:18.397333 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:18.397276 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" podStartSLOduration=1.704919774 podStartE2EDuration="6.39726632s" podCreationTimestamp="2026-06-03 16:26:12 +0000 UTC" firstStartedPulling="2026-06-03 16:26:13.018401491 +0000 UTC m=+303.282465344" lastFinishedPulling="2026-06-03 16:26:17.710748021 +0000 UTC m=+307.974811890" observedRunningTime="2026-06-03 16:26:18.395682857 +0000 UTC m=+308.659746733" watchObservedRunningTime="2026-06-03 16:26:18.39726632 +0000 UTC m=+308.661330194" Jun 03 16:26:29.373107 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:29.373079 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84984b75f8-dnlsx" Jun 03 16:26:31.297375 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.297331 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl"] Jun 03 16:26:31.307982 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.307953 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.317040 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.317022 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 03 16:26:31.318667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.318653 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 16:26:31.319503 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.319489 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 16:26:31.319800 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.319782 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 03 16:26:31.319832 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.319799 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w5kgr\"" Jun 03 16:26:31.343756 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.343732 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl"] Jun 03 16:26:31.409065 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.409030 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jslcq\" (UniqueName: \"kubernetes.io/projected/b775aa44-76cd-4941-9aff-8e54b4594a62-kube-api-access-jslcq\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.409240 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.409136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b775aa44-76cd-4941-9aff-8e54b4594a62-tmp\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.409311 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.409282 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b775aa44-76cd-4941-9aff-8e54b4594a62-tls-certs\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.510003 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.509971 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b775aa44-76cd-4941-9aff-8e54b4594a62-tls-certs\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.510003 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.510005 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jslcq\" (UniqueName: \"kubernetes.io/projected/b775aa44-76cd-4941-9aff-8e54b4594a62-kube-api-access-jslcq\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.510192 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.510036 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b775aa44-76cd-4941-9aff-8e54b4594a62-tmp\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.512343 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.512298 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b775aa44-76cd-4941-9aff-8e54b4594a62-tmp\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.512596 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.512575 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b775aa44-76cd-4941-9aff-8e54b4594a62-tls-certs\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.532479 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.532456 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jslcq\" (UniqueName: \"kubernetes.io/projected/b775aa44-76cd-4941-9aff-8e54b4594a62-kube-api-access-jslcq\") pod \"kube-auth-proxy-574ffd5976-wdnwl\" (UID: \"b775aa44-76cd-4941-9aff-8e54b4594a62\") " pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.617039 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.616969 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" Jun 03 16:26:31.630204 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.630164 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-spm6d"] Jun 03 16:26:31.633105 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.633088 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:31.635584 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.635564 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 03 16:26:31.635714 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.635598 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kgvtl\"" Jun 03 16:26:31.645672 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.645651 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-spm6d"] Jun 03 16:26:31.710905 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.710859 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwnh4\" (UniqueName: \"kubernetes.io/projected/ae134d74-aebf-4fb2-a309-d5eaf3f813a2-kube-api-access-gwnh4\") pod \"odh-model-controller-76588b95-spm6d\" (UID: \"ae134d74-aebf-4fb2-a309-d5eaf3f813a2\") " pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:31.711048 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.710909 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae134d74-aebf-4fb2-a309-d5eaf3f813a2-cert\") pod \"odh-model-controller-76588b95-spm6d\" (UID: \"ae134d74-aebf-4fb2-a309-d5eaf3f813a2\") " pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:31.760348 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.760277 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl"] Jun 03 16:26:31.763193 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:31.763159 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb775aa44_76cd_4941_9aff_8e54b4594a62.slice/crio-c02c8959fa9c8fdfa11f3fbc23e955e06fe8fbbc87a16cc2ad6028dbeb916236 WatchSource:0}: Error finding container c02c8959fa9c8fdfa11f3fbc23e955e06fe8fbbc87a16cc2ad6028dbeb916236: Status 404 returned error can't find the container with id c02c8959fa9c8fdfa11f3fbc23e955e06fe8fbbc87a16cc2ad6028dbeb916236 Jun 03 16:26:31.811312 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.811288 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae134d74-aebf-4fb2-a309-d5eaf3f813a2-cert\") pod \"odh-model-controller-76588b95-spm6d\" (UID: \"ae134d74-aebf-4fb2-a309-d5eaf3f813a2\") " pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:31.811438 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.811396 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gwnh4\" (UniqueName: \"kubernetes.io/projected/ae134d74-aebf-4fb2-a309-d5eaf3f813a2-kube-api-access-gwnh4\") pod \"odh-model-controller-76588b95-spm6d\" (UID: \"ae134d74-aebf-4fb2-a309-d5eaf3f813a2\") " pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:31.813682 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.813662 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae134d74-aebf-4fb2-a309-d5eaf3f813a2-cert\") pod \"odh-model-controller-76588b95-spm6d\" (UID: \"ae134d74-aebf-4fb2-a309-d5eaf3f813a2\") " pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:31.837383 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.837349 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwnh4\" (UniqueName: \"kubernetes.io/projected/ae134d74-aebf-4fb2-a309-d5eaf3f813a2-kube-api-access-gwnh4\") pod \"odh-model-controller-76588b95-spm6d\" (UID: \"ae134d74-aebf-4fb2-a309-d5eaf3f813a2\") " pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:31.948262 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:31.948168 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:32.101881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.101838 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-spm6d"] Jun 03 16:26:32.104637 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:32.104607 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae134d74_aebf_4fb2_a309_d5eaf3f813a2.slice/crio-d7771c07e64ab74ff55a0da6e41bab15439cb9625f28d9f8551a440b064b706f WatchSource:0}: Error finding container d7771c07e64ab74ff55a0da6e41bab15439cb9625f28d9f8551a440b064b706f: Status 404 returned error can't find the container with id d7771c07e64ab74ff55a0da6e41bab15439cb9625f28d9f8551a440b064b706f Jun 03 16:26:32.416153 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.416120 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-spm6d" event={"ID":"ae134d74-aebf-4fb2-a309-d5eaf3f813a2","Type":"ContainerStarted","Data":"d7771c07e64ab74ff55a0da6e41bab15439cb9625f28d9f8551a440b064b706f"} Jun 03 16:26:32.417092 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.417065 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" event={"ID":"b775aa44-76cd-4941-9aff-8e54b4594a62","Type":"ContainerStarted","Data":"c02c8959fa9c8fdfa11f3fbc23e955e06fe8fbbc87a16cc2ad6028dbeb916236"} Jun 03 16:26:32.710424 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.710338 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-6577596cd5-gflc2"] Jun 03 16:26:32.712961 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.712942 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:32.716010 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.715981 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 03 16:26:32.716119 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.716011 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-j9mjg\"" Jun 03 16:26:32.722039 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.721893 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-gflc2"] Jun 03 16:26:32.822639 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.819530 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmm7t\" (UniqueName: \"kubernetes.io/projected/85e2770d-ae55-4f53-bac5-8f860dffb76b-kube-api-access-pmm7t\") pod \"model-serving-api-6577596cd5-gflc2\" (UID: \"85e2770d-ae55-4f53-bac5-8f860dffb76b\") " pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:32.822639 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.819598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85e2770d-ae55-4f53-bac5-8f860dffb76b-tls-certs\") pod \"model-serving-api-6577596cd5-gflc2\" (UID: \"85e2770d-ae55-4f53-bac5-8f860dffb76b\") " pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:32.921779 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.920985 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pmm7t\" (UniqueName: \"kubernetes.io/projected/85e2770d-ae55-4f53-bac5-8f860dffb76b-kube-api-access-pmm7t\") pod \"model-serving-api-6577596cd5-gflc2\" (UID: \"85e2770d-ae55-4f53-bac5-8f860dffb76b\") " pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:32.921779 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.921040 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85e2770d-ae55-4f53-bac5-8f860dffb76b-tls-certs\") pod \"model-serving-api-6577596cd5-gflc2\" (UID: \"85e2770d-ae55-4f53-bac5-8f860dffb76b\") " pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:32.931376 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.931299 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85e2770d-ae55-4f53-bac5-8f860dffb76b-tls-certs\") pod \"model-serving-api-6577596cd5-gflc2\" (UID: \"85e2770d-ae55-4f53-bac5-8f860dffb76b\") " pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:32.939148 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:32.939118 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmm7t\" (UniqueName: \"kubernetes.io/projected/85e2770d-ae55-4f53-bac5-8f860dffb76b-kube-api-access-pmm7t\") pod \"model-serving-api-6577596cd5-gflc2\" (UID: \"85e2770d-ae55-4f53-bac5-8f860dffb76b\") " pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:33.026617 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.026581 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:33.232670 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.232640 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-gflc2"] Jun 03 16:26:33.233904 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:33.233868 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e2770d_ae55_4f53_bac5_8f860dffb76b.slice/crio-e2eea50380e0df82c1344fde9eeffefb8fab982e4ab7a980314d4cad5d9fce6a WatchSource:0}: Error finding container e2eea50380e0df82c1344fde9eeffefb8fab982e4ab7a980314d4cad5d9fce6a: Status 404 returned error can't find the container with id e2eea50380e0df82c1344fde9eeffefb8fab982e4ab7a980314d4cad5d9fce6a Jun 03 16:26:33.422811 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.422723 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-gflc2" event={"ID":"85e2770d-ae55-4f53-bac5-8f860dffb76b","Type":"ContainerStarted","Data":"e2eea50380e0df82c1344fde9eeffefb8fab982e4ab7a980314d4cad5d9fce6a"} Jun 03 16:26:33.476693 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.476609 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9"] Jun 03 16:26:33.480873 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.480839 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.483826 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.483783 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p5n4l\"" Jun 03 16:26:33.484105 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.484084 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:26:33.486859 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.486371 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:26:33.489366 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.488966 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9"] Jun 03 16:26:33.533649 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.533620 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbgqq\" (UniqueName: \"kubernetes.io/projected/ae36d147-c80a-4f53-b617-89409c82818c-kube-api-access-jbgqq\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.533797 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.533675 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.533797 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.533723 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.635188 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.635149 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.635378 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.635351 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.635579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.635552 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgqq\" (UniqueName: \"kubernetes.io/projected/ae36d147-c80a-4f53-b617-89409c82818c-kube-api-access-jbgqq\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.635688 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.635588 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.635803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.635695 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.645187 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.645138 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbgqq\" (UniqueName: \"kubernetes.io/projected/ae36d147-c80a-4f53-b617-89409c82818c-kube-api-access-jbgqq\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:33.797025 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:33.796990 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:34.416341 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:34.416287 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9"] Jun 03 16:26:35.085800 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:35.085761 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae36d147_c80a_4f53_b617_89409c82818c.slice/crio-25ce8ae2ffb91aa5d8d2aa28cc6539f696e5ca12d0299fbb1409059921b8bbd1 WatchSource:0}: Error finding container 25ce8ae2ffb91aa5d8d2aa28cc6539f696e5ca12d0299fbb1409059921b8bbd1: Status 404 returned error can't find the container with id 25ce8ae2ffb91aa5d8d2aa28cc6539f696e5ca12d0299fbb1409059921b8bbd1 Jun 03 16:26:35.434830 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:35.434749 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" event={"ID":"ae36d147-c80a-4f53-b617-89409c82818c","Type":"ContainerStarted","Data":"25ce8ae2ffb91aa5d8d2aa28cc6539f696e5ca12d0299fbb1409059921b8bbd1"} Jun 03 16:26:37.442966 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.442936 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-gflc2" event={"ID":"85e2770d-ae55-4f53-bac5-8f860dffb76b","Type":"ContainerStarted","Data":"5dc0633d8231a9d017c2aab537466909dd7212ae31dc55f4c657962d6634e9e9"} Jun 03 16:26:37.443387 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.443047 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:37.444525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.444491 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-spm6d" event={"ID":"ae134d74-aebf-4fb2-a309-d5eaf3f813a2","Type":"ContainerStarted","Data":"0c2c7e556b654d20fc1dc0ffd6f419d44605db8bd523d442d7fb745253206d5e"} Jun 03 16:26:37.444638 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.444584 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:37.445783 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.445757 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae36d147-c80a-4f53-b617-89409c82818c" containerID="289a576958ce9e3f2c4c42bdbdfc2ad9672f17685f0d456a9fe633440b63e4f4" exitCode=0 Jun 03 16:26:37.445874 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.445829 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" event={"ID":"ae36d147-c80a-4f53-b617-89409c82818c","Type":"ContainerDied","Data":"289a576958ce9e3f2c4c42bdbdfc2ad9672f17685f0d456a9fe633440b63e4f4"} Jun 03 16:26:37.447157 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.447138 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" event={"ID":"b775aa44-76cd-4941-9aff-8e54b4594a62","Type":"ContainerStarted","Data":"e494d2cb60534c0c7fc2af5e6ad57d66275c0779c506dea0cd8fcc5bbf740a90"} Jun 03 16:26:37.460673 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.460635 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/model-serving-api-6577596cd5-gflc2" podStartSLOduration=2.083184122 podStartE2EDuration="5.460623743s" podCreationTimestamp="2026-06-03 16:26:32 +0000 UTC" firstStartedPulling="2026-06-03 16:26:33.236500796 +0000 UTC m=+323.500564665" lastFinishedPulling="2026-06-03 16:26:36.613940429 +0000 UTC m=+326.878004286" observedRunningTime="2026-06-03 16:26:37.459490105 +0000 UTC m=+327.723553980" watchObservedRunningTime="2026-06-03 16:26:37.460623743 +0000 UTC m=+327.724687619" Jun 03 16:26:37.492437 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.492378 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-574ffd5976-wdnwl" podStartSLOduration=1.639961454 podStartE2EDuration="6.49236297s" podCreationTimestamp="2026-06-03 16:26:31 +0000 UTC" firstStartedPulling="2026-06-03 16:26:31.765372321 +0000 UTC m=+322.029436176" lastFinishedPulling="2026-06-03 16:26:36.617773836 +0000 UTC m=+326.881837692" observedRunningTime="2026-06-03 16:26:37.491688686 +0000 UTC m=+327.755752562" watchObservedRunningTime="2026-06-03 16:26:37.49236297 +0000 UTC m=+327.756426848" Jun 03 16:26:37.507762 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:37.507702 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-spm6d" podStartSLOduration=2.004850532 podStartE2EDuration="6.507687273s" podCreationTimestamp="2026-06-03 16:26:31 +0000 UTC" firstStartedPulling="2026-06-03 16:26:32.106043608 +0000 UTC m=+322.370107461" lastFinishedPulling="2026-06-03 16:26:36.608880347 +0000 UTC m=+326.872944202" observedRunningTime="2026-06-03 16:26:37.507546647 +0000 UTC m=+327.771610522" watchObservedRunningTime="2026-06-03 16:26:37.507687273 +0000 UTC m=+327.771751152" Jun 03 16:26:38.452398 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:38.452366 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae36d147-c80a-4f53-b617-89409c82818c" containerID="2386e924907aa4770acb3e97e11ab917d989047419dca06f4aa3bcb3716eda8f" exitCode=0 Jun 03 16:26:38.452799 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:38.452457 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" event={"ID":"ae36d147-c80a-4f53-b617-89409c82818c","Type":"ContainerDied","Data":"2386e924907aa4770acb3e97e11ab917d989047419dca06f4aa3bcb3716eda8f"} Jun 03 16:26:39.459044 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:39.459012 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae36d147-c80a-4f53-b617-89409c82818c" containerID="ddb3c1d06c54bb91ba38ccc89b504e0f93f7c68d1e16dcb76c7057d4f7d15bc8" exitCode=0 Jun 03 16:26:39.459404 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:39.459090 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" event={"ID":"ae36d147-c80a-4f53-b617-89409c82818c","Type":"ContainerDied","Data":"ddb3c1d06c54bb91ba38ccc89b504e0f93f7c68d1e16dcb76c7057d4f7d15bc8"} Jun 03 16:26:40.584015 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.583989 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:40.704708 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.704681 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-bundle\") pod \"ae36d147-c80a-4f53-b617-89409c82818c\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " Jun 03 16:26:40.704852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.704767 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbgqq\" (UniqueName: \"kubernetes.io/projected/ae36d147-c80a-4f53-b617-89409c82818c-kube-api-access-jbgqq\") pod \"ae36d147-c80a-4f53-b617-89409c82818c\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " Jun 03 16:26:40.704852 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.704821 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-util\") pod \"ae36d147-c80a-4f53-b617-89409c82818c\" (UID: \"ae36d147-c80a-4f53-b617-89409c82818c\") " Jun 03 16:26:40.705617 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.705588 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-bundle" (OuterVolumeSpecName: "bundle") pod "ae36d147-c80a-4f53-b617-89409c82818c" (UID: "ae36d147-c80a-4f53-b617-89409c82818c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:26:40.706793 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.706772 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae36d147-c80a-4f53-b617-89409c82818c-kube-api-access-jbgqq" (OuterVolumeSpecName: "kube-api-access-jbgqq") pod "ae36d147-c80a-4f53-b617-89409c82818c" (UID: "ae36d147-c80a-4f53-b617-89409c82818c"). InnerVolumeSpecName "kube-api-access-jbgqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:26:40.712426 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.712370 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-util" (OuterVolumeSpecName: "util") pod "ae36d147-c80a-4f53-b617-89409c82818c" (UID: "ae36d147-c80a-4f53-b617-89409c82818c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:26:40.806119 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.806095 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbgqq\" (UniqueName: \"kubernetes.io/projected/ae36d147-c80a-4f53-b617-89409c82818c-kube-api-access-jbgqq\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:40.806119 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.806117 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:40.806246 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:40.806127 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae36d147-c80a-4f53-b617-89409c82818c-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:41.467336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.467297 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" event={"ID":"ae36d147-c80a-4f53-b617-89409c82818c","Type":"ContainerDied","Data":"25ce8ae2ffb91aa5d8d2aa28cc6539f696e5ca12d0299fbb1409059921b8bbd1"} Jun 03 16:26:41.467336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.467313 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835rc7f9" Jun 03 16:26:41.467336 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.467342 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25ce8ae2ffb91aa5d8d2aa28cc6539f696e5ca12d0299fbb1409059921b8bbd1" Jun 03 16:26:41.942449 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.942413 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm"] Jun 03 16:26:41.942795 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.942756 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae36d147-c80a-4f53-b617-89409c82818c" containerName="pull" Jun 03 16:26:41.942795 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.942767 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae36d147-c80a-4f53-b617-89409c82818c" containerName="pull" Jun 03 16:26:41.942795 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.942779 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae36d147-c80a-4f53-b617-89409c82818c" containerName="extract" Jun 03 16:26:41.942795 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.942785 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae36d147-c80a-4f53-b617-89409c82818c" containerName="extract" Jun 03 16:26:41.942795 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.942793 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae36d147-c80a-4f53-b617-89409c82818c" containerName="util" Jun 03 16:26:41.942946 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.942799 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae36d147-c80a-4f53-b617-89409c82818c" containerName="util" Jun 03 16:26:41.942946 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.942856 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae36d147-c80a-4f53-b617-89409c82818c" containerName="extract" Jun 03 16:26:41.946722 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.946699 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:41.949120 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.949099 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 03 16:26:41.949589 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.949575 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2fjhg\"" Jun 03 16:26:41.960553 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.960530 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp"] Jun 03 16:26:41.963834 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.963816 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm"] Jun 03 16:26:41.963934 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.963902 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:41.966970 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.966950 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mprj7\"" Jun 03 16:26:41.967064 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.966999 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 03 16:26:41.982247 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:41.982217 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp"] Jun 03 16:26:42.016186 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.016156 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-888jz\" (UniqueName: \"kubernetes.io/projected/db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1-kube-api-access-888jz\") pod \"kserve-controller-manager-7c99ffcdcd-5tckm\" (UID: \"db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:42.016363 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.016261 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1-cert\") pod \"kserve-controller-manager-7c99ffcdcd-5tckm\" (UID: \"db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:42.117235 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.117203 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1-cert\") pod \"kserve-controller-manager-7c99ffcdcd-5tckm\" (UID: \"db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:42.117401 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.117247 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-888jz\" (UniqueName: \"kubernetes.io/projected/db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1-kube-api-access-888jz\") pod \"kserve-controller-manager-7c99ffcdcd-5tckm\" (UID: \"db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:42.117401 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.117283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23fc4020-dd85-4e8f-a131-834db42b1559-cert\") pod \"llmisvc-controller-manager-c6b9847d7-bdpsp\" (UID: \"23fc4020-dd85-4e8f-a131-834db42b1559\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:42.117401 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.117361 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87wjg\" (UniqueName: \"kubernetes.io/projected/23fc4020-dd85-4e8f-a131-834db42b1559-kube-api-access-87wjg\") pod \"llmisvc-controller-manager-c6b9847d7-bdpsp\" (UID: \"23fc4020-dd85-4e8f-a131-834db42b1559\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:42.119566 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.119538 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1-cert\") pod \"kserve-controller-manager-7c99ffcdcd-5tckm\" (UID: \"db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:42.129179 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.129157 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-888jz\" (UniqueName: \"kubernetes.io/projected/db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1-kube-api-access-888jz\") pod \"kserve-controller-manager-7c99ffcdcd-5tckm\" (UID: \"db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:42.218460 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.218396 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-87wjg\" (UniqueName: \"kubernetes.io/projected/23fc4020-dd85-4e8f-a131-834db42b1559-kube-api-access-87wjg\") pod \"llmisvc-controller-manager-c6b9847d7-bdpsp\" (UID: \"23fc4020-dd85-4e8f-a131-834db42b1559\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:42.218460 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.218456 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23fc4020-dd85-4e8f-a131-834db42b1559-cert\") pod \"llmisvc-controller-manager-c6b9847d7-bdpsp\" (UID: \"23fc4020-dd85-4e8f-a131-834db42b1559\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:42.220618 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.220599 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23fc4020-dd85-4e8f-a131-834db42b1559-cert\") pod \"llmisvc-controller-manager-c6b9847d7-bdpsp\" (UID: \"23fc4020-dd85-4e8f-a131-834db42b1559\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:42.234943 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.234918 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87wjg\" (UniqueName: \"kubernetes.io/projected/23fc4020-dd85-4e8f-a131-834db42b1559-kube-api-access-87wjg\") pod \"llmisvc-controller-manager-c6b9847d7-bdpsp\" (UID: \"23fc4020-dd85-4e8f-a131-834db42b1559\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:42.256953 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.256932 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:42.273617 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.273596 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:42.394649 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.394615 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm"] Jun 03 16:26:42.396345 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:42.396306 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2c46c0_3e4f_4ac5_80bd_f660ccf5dbb1.slice/crio-3ca70be12b4fdd869365283fe9be719a8f53fa25278cc061a4eb063c6193128a WatchSource:0}: Error finding container 3ca70be12b4fdd869365283fe9be719a8f53fa25278cc061a4eb063c6193128a: Status 404 returned error can't find the container with id 3ca70be12b4fdd869365283fe9be719a8f53fa25278cc061a4eb063c6193128a Jun 03 16:26:42.416616 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.416561 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp"] Jun 03 16:26:42.418754 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:42.418725 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod23fc4020_dd85_4e8f_a131_834db42b1559.slice/crio-645316176c2d73911159abb63b1d2f8ecf3f8be9e48ea2075f1aaae8e7baddef WatchSource:0}: Error finding container 645316176c2d73911159abb63b1d2f8ecf3f8be9e48ea2075f1aaae8e7baddef: Status 404 returned error can't find the container with id 645316176c2d73911159abb63b1d2f8ecf3f8be9e48ea2075f1aaae8e7baddef Jun 03 16:26:42.472547 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.472466 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" event={"ID":"db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1","Type":"ContainerStarted","Data":"3ca70be12b4fdd869365283fe9be719a8f53fa25278cc061a4eb063c6193128a"} Jun 03 16:26:42.473604 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:42.473585 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" event={"ID":"23fc4020-dd85-4e8f-a131-834db42b1559","Type":"ContainerStarted","Data":"645316176c2d73911159abb63b1d2f8ecf3f8be9e48ea2075f1aaae8e7baddef"} Jun 03 16:26:45.488690 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:45.488597 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" event={"ID":"db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1","Type":"ContainerStarted","Data":"a7e58caa1f2516684daeb1124f804f50665e7759145c4494eada7bab92e6be86"} Jun 03 16:26:45.489103 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:45.488806 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:26:45.527115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:45.527059 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" podStartSLOduration=1.842677159 podStartE2EDuration="4.527043386s" podCreationTimestamp="2026-06-03 16:26:41 +0000 UTC" firstStartedPulling="2026-06-03 16:26:42.398115116 +0000 UTC m=+332.662178969" lastFinishedPulling="2026-06-03 16:26:45.082481329 +0000 UTC m=+335.346545196" observedRunningTime="2026-06-03 16:26:45.526162315 +0000 UTC m=+335.790226190" watchObservedRunningTime="2026-06-03 16:26:45.527043386 +0000 UTC m=+335.791107259" Jun 03 16:26:46.493366 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:46.493312 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" event={"ID":"23fc4020-dd85-4e8f-a131-834db42b1559","Type":"ContainerStarted","Data":"7259a5206f89b15b588306998698ddb76c162a8e39fff6900b0fc6125249f9c5"} Jun 03 16:26:46.514946 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:46.514900 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" podStartSLOduration=2.011572515 podStartE2EDuration="5.514887622s" podCreationTimestamp="2026-06-03 16:26:41 +0000 UTC" firstStartedPulling="2026-06-03 16:26:42.420103299 +0000 UTC m=+332.684167153" lastFinishedPulling="2026-06-03 16:26:45.923418405 +0000 UTC m=+336.187482260" observedRunningTime="2026-06-03 16:26:46.51294468 +0000 UTC m=+336.777008556" watchObservedRunningTime="2026-06-03 16:26:46.514887622 +0000 UTC m=+336.778951496" Jun 03 16:26:47.496827 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:47.496753 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:26:48.455095 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:48.455017 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-spm6d" Jun 03 16:26:48.457946 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:48.457922 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-6577596cd5-gflc2" Jun 03 16:26:49.323006 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.322975 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg"] Jun 03 16:26:49.326533 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.326512 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.330771 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.330750 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:26:49.331554 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.331533 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p5n4l\"" Jun 03 16:26:49.332345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.332311 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:26:49.344841 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.344817 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg"] Jun 03 16:26:49.486807 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.486770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.486807 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.486822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp5d5\" (UniqueName: \"kubernetes.io/projected/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-kube-api-access-pp5d5\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.487004 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.486919 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.587437 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.587347 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.587437 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.587407 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pp5d5\" (UniqueName: \"kubernetes.io/projected/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-kube-api-access-pp5d5\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.587629 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.587469 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.587828 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.587810 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.587871 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.587827 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.612661 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.612637 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp5d5\" (UniqueName: \"kubernetes.io/projected/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-kube-api-access-pp5d5\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.637939 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.637915 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:49.781789 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:49.781754 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c94ff63_322e_4dfe_9fbd_96ad3c69b8c0.slice/crio-48a1297338d0332ed64215f3fee896e16a34042d241a8febff41133a69de0528 WatchSource:0}: Error finding container 48a1297338d0332ed64215f3fee896e16a34042d241a8febff41133a69de0528: Status 404 returned error can't find the container with id 48a1297338d0332ed64215f3fee896e16a34042d241a8febff41133a69de0528 Jun 03 16:26:49.781897 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:49.781874 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg"] Jun 03 16:26:50.508888 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.508839 2576 generic.go:358] "Generic (PLEG): container finished" podID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerID="9ab34103bbc544e9e55c5c4aeed12023deb675809171130c8c385f66130c1f5b" exitCode=0 Jun 03 16:26:50.509245 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.508918 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" event={"ID":"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0","Type":"ContainerDied","Data":"9ab34103bbc544e9e55c5c4aeed12023deb675809171130c8c385f66130c1f5b"} Jun 03 16:26:50.509245 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.508946 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" event={"ID":"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0","Type":"ContainerStarted","Data":"48a1297338d0332ed64215f3fee896e16a34042d241a8febff41133a69de0528"} Jun 03 16:26:50.792006 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.791931 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj"] Jun 03 16:26:50.795420 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.795394 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:50.799833 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.799811 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 03 16:26:50.799948 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.799850 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 03 16:26:50.800175 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.800156 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-x4d57\"" Jun 03 16:26:50.811466 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.811419 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj"] Jun 03 16:26:50.896977 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.896920 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wxg9\" (UniqueName: \"kubernetes.io/projected/49c66fa3-d643-4f5d-9c57-d23370984f6b-kube-api-access-5wxg9\") pod \"servicemesh-operator3-55f49c5f94-6qrjj\" (UID: \"49c66fa3-d643-4f5d-9c57-d23370984f6b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:50.897127 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.897032 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/49c66fa3-d643-4f5d-9c57-d23370984f6b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-6qrjj\" (UID: \"49c66fa3-d643-4f5d-9c57-d23370984f6b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:50.998187 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.998151 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5wxg9\" (UniqueName: \"kubernetes.io/projected/49c66fa3-d643-4f5d-9c57-d23370984f6b-kube-api-access-5wxg9\") pod \"servicemesh-operator3-55f49c5f94-6qrjj\" (UID: \"49c66fa3-d643-4f5d-9c57-d23370984f6b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:50.998363 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:50.998214 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/49c66fa3-d643-4f5d-9c57-d23370984f6b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-6qrjj\" (UID: \"49c66fa3-d643-4f5d-9c57-d23370984f6b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:51.000944 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:51.000912 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/49c66fa3-d643-4f5d-9c57-d23370984f6b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-6qrjj\" (UID: \"49c66fa3-d643-4f5d-9c57-d23370984f6b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:51.010128 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:51.010104 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wxg9\" (UniqueName: \"kubernetes.io/projected/49c66fa3-d643-4f5d-9c57-d23370984f6b-kube-api-access-5wxg9\") pod \"servicemesh-operator3-55f49c5f94-6qrjj\" (UID: \"49c66fa3-d643-4f5d-9c57-d23370984f6b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:51.105561 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:51.105495 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:51.251884 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:51.251862 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj"] Jun 03 16:26:51.255082 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:26:51.255060 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c66fa3_d643_4f5d_9c57_d23370984f6b.slice/crio-b173430147fae25c1c8b478824bd9659bfa82fda7f48a896dee7b12781d67787 WatchSource:0}: Error finding container b173430147fae25c1c8b478824bd9659bfa82fda7f48a896dee7b12781d67787: Status 404 returned error can't find the container with id b173430147fae25c1c8b478824bd9659bfa82fda7f48a896dee7b12781d67787 Jun 03 16:26:51.514579 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:51.514550 2576 generic.go:358] "Generic (PLEG): container finished" podID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerID="c8b4576f50aa9a7d7790cf668332951a02a3705e500501eedf9ef8210f3d29b9" exitCode=0 Jun 03 16:26:51.514918 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:51.514613 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" event={"ID":"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0","Type":"ContainerDied","Data":"c8b4576f50aa9a7d7790cf668332951a02a3705e500501eedf9ef8210f3d29b9"} Jun 03 16:26:51.515883 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:51.515858 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" event={"ID":"49c66fa3-d643-4f5d-9c57-d23370984f6b","Type":"ContainerStarted","Data":"b173430147fae25c1c8b478824bd9659bfa82fda7f48a896dee7b12781d67787"} Jun 03 16:26:52.522174 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:52.522132 2576 generic.go:358] "Generic (PLEG): container finished" podID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerID="721f0d58eb3c9377f60090667eee7ce5c5237c7dcede521078a1e02a40ef3d27" exitCode=0 Jun 03 16:26:52.522576 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:52.522210 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" event={"ID":"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0","Type":"ContainerDied","Data":"721f0d58eb3c9377f60090667eee7ce5c5237c7dcede521078a1e02a40ef3d27"} Jun 03 16:26:53.529696 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.529660 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" event={"ID":"49c66fa3-d643-4f5d-9c57-d23370984f6b","Type":"ContainerStarted","Data":"3064e8f58cbc67c5d634a74f28096e5f757a267f2dd9b6cf5b9cb764ceefd32b"} Jun 03 16:26:53.530265 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.530238 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:26:53.571031 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.570971 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" podStartSLOduration=1.445321139 podStartE2EDuration="3.570955701s" podCreationTimestamp="2026-06-03 16:26:50 +0000 UTC" firstStartedPulling="2026-06-03 16:26:51.258473502 +0000 UTC m=+341.522537355" lastFinishedPulling="2026-06-03 16:26:53.38410806 +0000 UTC m=+343.648171917" observedRunningTime="2026-06-03 16:26:53.567791891 +0000 UTC m=+343.831855793" watchObservedRunningTime="2026-06-03 16:26:53.570955701 +0000 UTC m=+343.835019576" Jun 03 16:26:53.687168 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.687146 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:53.824523 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.824495 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-util\") pod \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " Jun 03 16:26:53.824689 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.824552 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp5d5\" (UniqueName: \"kubernetes.io/projected/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-kube-api-access-pp5d5\") pod \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " Jun 03 16:26:53.824689 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.824586 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-bundle\") pod \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\" (UID: \"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0\") " Jun 03 16:26:53.825459 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.825431 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-bundle" (OuterVolumeSpecName: "bundle") pod "8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" (UID: "8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:26:53.826738 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.826711 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-kube-api-access-pp5d5" (OuterVolumeSpecName: "kube-api-access-pp5d5") pod "8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" (UID: "8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0"). InnerVolumeSpecName "kube-api-access-pp5d5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:26:53.829910 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.829888 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-util" (OuterVolumeSpecName: "util") pod "8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" (UID: "8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:26:53.925546 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.925501 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp5d5\" (UniqueName: \"kubernetes.io/projected/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-kube-api-access-pp5d5\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:53.925546 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.925521 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:53.925546 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:53.925534 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:26:54.535703 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:54.535674 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" Jun 03 16:26:54.536032 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:54.535673 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awxrjg" event={"ID":"8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0","Type":"ContainerDied","Data":"48a1297338d0332ed64215f3fee896e16a34042d241a8febff41133a69de0528"} Jun 03 16:26:54.536032 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:26:54.535805 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48a1297338d0332ed64215f3fee896e16a34042d241a8febff41133a69de0528" Jun 03 16:27:01.079795 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.079759 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8"] Jun 03 16:27:01.080271 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.080253 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerName="util" Jun 03 16:27:01.080375 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.080274 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerName="util" Jun 03 16:27:01.080375 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.080290 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerName="pull" Jun 03 16:27:01.080375 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.080298 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerName="pull" Jun 03 16:27:01.080375 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.080354 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerName="extract" Jun 03 16:27:01.080375 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.080363 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerName="extract" Jun 03 16:27:01.080620 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.080454 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c94ff63-322e-4dfe-9fbd-96ad3c69b8c0" containerName="extract" Jun 03 16:27:01.084458 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.084438 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.087396 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.087380 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 03 16:27:01.087546 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.087525 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ttsl6\"" Jun 03 16:27:01.087654 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.087595 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 03 16:27:01.087711 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.087665 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 03 16:27:01.087765 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.087667 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 03 16:27:01.094252 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.094228 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8"] Jun 03 16:27:01.183133 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.183106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57db75db-7542-4bc3-87e0-1367439c8160-istio-token\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.183281 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.183150 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57db75db-7542-4bc3-87e0-1367439c8160-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.183281 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.183178 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.183281 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.183199 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpqk6\" (UniqueName: \"kubernetes.io/projected/57db75db-7542-4bc3-87e0-1367439c8160-kube-api-access-vpqk6\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.183281 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.183218 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-cacerts\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.183281 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.183235 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57db75db-7542-4bc3-87e0-1367439c8160-local-certs\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.183463 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.183276 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.284519 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.284487 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-cacerts\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.284519 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.284521 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57db75db-7542-4bc3-87e0-1367439c8160-local-certs\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.284731 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.284549 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.284731 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.284602 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57db75db-7542-4bc3-87e0-1367439c8160-istio-token\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.284731 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.284644 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57db75db-7542-4bc3-87e0-1367439c8160-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.284731 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.284686 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.284731 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.284716 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vpqk6\" (UniqueName: \"kubernetes.io/projected/57db75db-7542-4bc3-87e0-1367439c8160-kube-api-access-vpqk6\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.285340 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.285295 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57db75db-7542-4bc3-87e0-1367439c8160-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.287142 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.287016 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.287142 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.287099 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-cacerts\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.287538 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.287519 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57db75db-7542-4bc3-87e0-1367439c8160-local-certs\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.287785 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.287765 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57db75db-7542-4bc3-87e0-1367439c8160-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.295987 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.295953 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpqk6\" (UniqueName: \"kubernetes.io/projected/57db75db-7542-4bc3-87e0-1367439c8160-kube-api-access-vpqk6\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.296681 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.296659 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57db75db-7542-4bc3-87e0-1367439c8160-istio-token\") pod \"istiod-openshift-gateway-798958bb55-qvwz8\" (UID: \"57db75db-7542-4bc3-87e0-1367439c8160\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.394102 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.394032 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:01.535763 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.535730 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-5c5d698fd4-8tfkd"] Jun 03 16:27:01.540535 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.540513 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" Jun 03 16:27:01.540794 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.540695 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8"] Jun 03 16:27:01.542957 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:27:01.542928 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57db75db_7542_4bc3_87e0_1367439c8160.slice/crio-1a9d6b656cf69fb54d5f888729b275a1d0c83566cb1d921161911ec1881e605e WatchSource:0}: Error finding container 1a9d6b656cf69fb54d5f888729b275a1d0c83566cb1d921161911ec1881e605e: Status 404 returned error can't find the container with id 1a9d6b656cf69fb54d5f888729b275a1d0c83566cb1d921161911ec1881e605e Jun 03 16:27:01.543655 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.543603 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-b864n\"" Jun 03 16:27:01.543743 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.543633 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 03 16:27:01.550621 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.550600 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-5c5d698fd4-8tfkd"] Jun 03 16:27:01.562017 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.561993 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" event={"ID":"57db75db-7542-4bc3-87e0-1367439c8160","Type":"ContainerStarted","Data":"1a9d6b656cf69fb54d5f888729b275a1d0c83566cb1d921161911ec1881e605e"} Jun 03 16:27:01.587569 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.587547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwp5l\" (UniqueName: \"kubernetes.io/projected/815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834-kube-api-access-vwp5l\") pod \"dashboard-redirect-5c5d698fd4-8tfkd\" (UID: \"815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834\") " pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" Jun 03 16:27:01.587689 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.587596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834-redirect-config\") pod \"dashboard-redirect-5c5d698fd4-8tfkd\" (UID: \"815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834\") " pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" Jun 03 16:27:01.688617 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.688535 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vwp5l\" (UniqueName: \"kubernetes.io/projected/815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834-kube-api-access-vwp5l\") pod \"dashboard-redirect-5c5d698fd4-8tfkd\" (UID: \"815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834\") " pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" Jun 03 16:27:01.688617 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.688590 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834-redirect-config\") pod \"dashboard-redirect-5c5d698fd4-8tfkd\" (UID: \"815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834\") " pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" Jun 03 16:27:01.689380 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.689357 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834-redirect-config\") pod \"dashboard-redirect-5c5d698fd4-8tfkd\" (UID: \"815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834\") " pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" Jun 03 16:27:01.696525 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.696501 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwp5l\" (UniqueName: \"kubernetes.io/projected/815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834-kube-api-access-vwp5l\") pod \"dashboard-redirect-5c5d698fd4-8tfkd\" (UID: \"815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834\") " pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" Jun 03 16:27:01.863394 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.863358 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" Jun 03 16:27:01.992664 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:01.992636 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-5c5d698fd4-8tfkd"] Jun 03 16:27:01.994161 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:27:01.994135 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815ec6f4_ff7c_4e1a_8576_3ea9bcf6c834.slice/crio-c7ba41f584e9385943cf23bb3e12ddba6e20388fef2c5e75c0f8f47ec3266c89 WatchSource:0}: Error finding container c7ba41f584e9385943cf23bb3e12ddba6e20388fef2c5e75c0f8f47ec3266c89: Status 404 returned error can't find the container with id c7ba41f584e9385943cf23bb3e12ddba6e20388fef2c5e75c0f8f47ec3266c89 Jun 03 16:27:02.574335 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:02.574292 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" event={"ID":"815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834","Type":"ContainerStarted","Data":"c7ba41f584e9385943cf23bb3e12ddba6e20388fef2c5e75c0f8f47ec3266c89"} Jun 03 16:27:05.541785 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:05.541756 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-6qrjj" Jun 03 16:27:07.216043 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:07.215997 2576 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 16:27:07.216372 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:07.216071 2576 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 03 16:27:07.597833 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:07.597798 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" event={"ID":"57db75db-7542-4bc3-87e0-1367439c8160","Type":"ContainerStarted","Data":"f5c6e20c825e61304ef459180b670c34a974a5799892be808abd7d0318496af0"} Jun 03 16:27:07.598066 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:07.598037 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:07.599218 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:07.599198 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" event={"ID":"815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834","Type":"ContainerStarted","Data":"9c66492dfc44f3040bb7e957ad664d5a8322bf6b8206b366534e887e12f833d3"} Jun 03 16:27:07.631500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:07.631462 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" podStartSLOduration=0.961663227 podStartE2EDuration="6.631451414s" podCreationTimestamp="2026-06-03 16:27:01 +0000 UTC" firstStartedPulling="2026-06-03 16:27:01.545955975 +0000 UTC m=+351.810019833" lastFinishedPulling="2026-06-03 16:27:07.215744154 +0000 UTC m=+357.479808020" observedRunningTime="2026-06-03 16:27:07.628985142 +0000 UTC m=+357.893049019" watchObservedRunningTime="2026-06-03 16:27:07.631451414 +0000 UTC m=+357.895515289" Jun 03 16:27:07.648183 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:07.648144 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-5c5d698fd4-8tfkd" podStartSLOduration=1.3667718469999999 podStartE2EDuration="6.64812934s" podCreationTimestamp="2026-06-03 16:27:01 +0000 UTC" firstStartedPulling="2026-06-03 16:27:01.995470094 +0000 UTC m=+352.259533947" lastFinishedPulling="2026-06-03 16:27:07.276827527 +0000 UTC m=+357.540891440" observedRunningTime="2026-06-03 16:27:07.646232842 +0000 UTC m=+357.910296716" watchObservedRunningTime="2026-06-03 16:27:07.64812934 +0000 UTC m=+357.912193217" Jun 03 16:27:08.605474 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:08.605439 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-qvwz8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 03 16:27:08.605816 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:08.605492 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" podUID="57db75db-7542-4bc3-87e0-1367439c8160" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:27:11.604636 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:11.604606 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qvwz8" Jun 03 16:27:16.498799 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:16.498771 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-5tckm" Jun 03 16:27:18.503010 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:18.502976 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-bdpsp" Jun 03 16:27:32.553231 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.553191 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2tgzj"] Jun 03 16:27:32.560858 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.560837 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" Jun 03 16:27:32.563667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.563637 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 16:27:32.563667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.563655 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 16:27:32.564529 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.564513 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-857v5\"" Jun 03 16:27:32.570674 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.570651 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2tgzj"] Jun 03 16:27:32.637472 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.637447 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxl74\" (UniqueName: \"kubernetes.io/projected/324dacac-ada1-4bc3-b17c-fbe7e13c8af0-kube-api-access-sxl74\") pod \"kuadrant-operator-catalog-2tgzj\" (UID: \"324dacac-ada1-4bc3-b17c-fbe7e13c8af0\") " pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" Jun 03 16:27:32.738856 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.738827 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sxl74\" (UniqueName: \"kubernetes.io/projected/324dacac-ada1-4bc3-b17c-fbe7e13c8af0-kube-api-access-sxl74\") pod \"kuadrant-operator-catalog-2tgzj\" (UID: \"324dacac-ada1-4bc3-b17c-fbe7e13c8af0\") " pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" Jun 03 16:27:32.747090 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.747061 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxl74\" (UniqueName: \"kubernetes.io/projected/324dacac-ada1-4bc3-b17c-fbe7e13c8af0-kube-api-access-sxl74\") pod \"kuadrant-operator-catalog-2tgzj\" (UID: \"324dacac-ada1-4bc3-b17c-fbe7e13c8af0\") " pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" Jun 03 16:27:32.871357 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.871264 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" Jun 03 16:27:32.921557 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:32.921529 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2tgzj"] Jun 03 16:27:33.002541 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:33.002446 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2tgzj"] Jun 03 16:27:33.004935 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:27:33.004909 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod324dacac_ada1_4bc3_b17c_fbe7e13c8af0.slice/crio-659654b2856f7f7472666f0fb1d3b3f66b4833d75bc5fe00556acc323bc29253 WatchSource:0}: Error finding container 659654b2856f7f7472666f0fb1d3b3f66b4833d75bc5fe00556acc323bc29253: Status 404 returned error can't find the container with id 659654b2856f7f7472666f0fb1d3b3f66b4833d75bc5fe00556acc323bc29253 Jun 03 16:27:33.694739 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:33.694704 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" event={"ID":"324dacac-ada1-4bc3-b17c-fbe7e13c8af0","Type":"ContainerStarted","Data":"659654b2856f7f7472666f0fb1d3b3f66b4833d75bc5fe00556acc323bc29253"} Jun 03 16:27:35.704555 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:35.704522 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" event={"ID":"324dacac-ada1-4bc3-b17c-fbe7e13c8af0","Type":"ContainerStarted","Data":"e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc"} Jun 03 16:27:35.704956 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:35.704572 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" podUID="324dacac-ada1-4bc3-b17c-fbe7e13c8af0" containerName="registry-server" containerID="cri-o://e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc" gracePeriod=2 Jun 03 16:27:35.729020 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:35.728965 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" podStartSLOduration=1.8267238670000001 podStartE2EDuration="3.728951597s" podCreationTimestamp="2026-06-03 16:27:32 +0000 UTC" firstStartedPulling="2026-06-03 16:27:33.006181089 +0000 UTC m=+383.270244942" lastFinishedPulling="2026-06-03 16:27:34.908408816 +0000 UTC m=+385.172472672" observedRunningTime="2026-06-03 16:27:35.727031402 +0000 UTC m=+385.991095278" watchObservedRunningTime="2026-06-03 16:27:35.728951597 +0000 UTC m=+385.993015471" Jun 03 16:27:35.958411 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:35.958356 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" Jun 03 16:27:36.069247 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.069209 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxl74\" (UniqueName: \"kubernetes.io/projected/324dacac-ada1-4bc3-b17c-fbe7e13c8af0-kube-api-access-sxl74\") pod \"324dacac-ada1-4bc3-b17c-fbe7e13c8af0\" (UID: \"324dacac-ada1-4bc3-b17c-fbe7e13c8af0\") " Jun 03 16:27:36.071343 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.071304 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/324dacac-ada1-4bc3-b17c-fbe7e13c8af0-kube-api-access-sxl74" (OuterVolumeSpecName: "kube-api-access-sxl74") pod "324dacac-ada1-4bc3-b17c-fbe7e13c8af0" (UID: "324dacac-ada1-4bc3-b17c-fbe7e13c8af0"). InnerVolumeSpecName "kube-api-access-sxl74". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:27:36.170207 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.170170 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxl74\" (UniqueName: \"kubernetes.io/projected/324dacac-ada1-4bc3-b17c-fbe7e13c8af0-kube-api-access-sxl74\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:36.709149 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.709069 2576 generic.go:358] "Generic (PLEG): container finished" podID="324dacac-ada1-4bc3-b17c-fbe7e13c8af0" containerID="e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc" exitCode=0 Jun 03 16:27:36.709149 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.709107 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" event={"ID":"324dacac-ada1-4bc3-b17c-fbe7e13c8af0","Type":"ContainerDied","Data":"e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc"} Jun 03 16:27:36.709149 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.709133 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" event={"ID":"324dacac-ada1-4bc3-b17c-fbe7e13c8af0","Type":"ContainerDied","Data":"659654b2856f7f7472666f0fb1d3b3f66b4833d75bc5fe00556acc323bc29253"} Jun 03 16:27:36.709682 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.709157 2576 scope.go:117] "RemoveContainer" containerID="e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc" Jun 03 16:27:36.709682 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.709162 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2tgzj" Jun 03 16:27:36.717935 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.717904 2576 scope.go:117] "RemoveContainer" containerID="e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc" Jun 03 16:27:36.718169 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:27:36.718151 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc\": container with ID starting with e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc not found: ID does not exist" containerID="e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc" Jun 03 16:27:36.718225 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.718180 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc"} err="failed to get container status \"e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc\": rpc error: code = NotFound desc = could not find container \"e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc\": container with ID starting with e66cb2e3654b54047627a17af532af9dddc1f0112600ce7eaa4f163bf39803cc not found: ID does not exist" Jun 03 16:27:36.725730 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.725703 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2tgzj"] Jun 03 16:27:36.729957 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:36.729937 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2tgzj"] Jun 03 16:27:38.313247 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:38.313213 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="324dacac-ada1-4bc3-b17c-fbe7e13c8af0" path="/var/lib/kubelet/pods/324dacac-ada1-4bc3-b17c-fbe7e13c8af0/volumes" Jun 03 16:27:48.571686 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.571648 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc"] Jun 03 16:27:48.572220 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.572203 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="324dacac-ada1-4bc3-b17c-fbe7e13c8af0" containerName="registry-server" Jun 03 16:27:48.572264 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.572223 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="324dacac-ada1-4bc3-b17c-fbe7e13c8af0" containerName="registry-server" Jun 03 16:27:48.572362 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.572349 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="324dacac-ada1-4bc3-b17c-fbe7e13c8af0" containerName="registry-server" Jun 03 16:27:48.575919 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.575898 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.578702 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.578678 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 16:27:48.578702 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.578693 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 16:27:48.579513 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.579489 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7w95z\"" Jun 03 16:27:48.583554 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.583512 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc"] Jun 03 16:27:48.665512 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.665487 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.665763 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.665538 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cj7r\" (UniqueName: \"kubernetes.io/projected/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-kube-api-access-4cj7r\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.665763 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.665580 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.766659 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.766635 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4cj7r\" (UniqueName: \"kubernetes.io/projected/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-kube-api-access-4cj7r\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.766753 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.766663 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.766753 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.766703 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.766990 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.766973 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.767032 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.767005 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.775920 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.775900 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cj7r\" (UniqueName: \"kubernetes.io/projected/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-kube-api-access-4cj7r\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:48.887006 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:48.886947 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:49.016875 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.016855 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc"] Jun 03 16:27:49.018771 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:27:49.018738 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5a9e07_f2d2_4d8b_a73e_64b82b2c9595.slice/crio-0ef52ea4ce0c2f659b381811412c00023cc902fbdcc757bd7f64fd4f44c1ed5b WatchSource:0}: Error finding container 0ef52ea4ce0c2f659b381811412c00023cc902fbdcc757bd7f64fd4f44c1ed5b: Status 404 returned error can't find the container with id 0ef52ea4ce0c2f659b381811412c00023cc902fbdcc757bd7f64fd4f44c1ed5b Jun 03 16:27:49.167811 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.167758 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5"] Jun 03 16:27:49.171594 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.171559 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.180971 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.180951 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5"] Jun 03 16:27:49.271292 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.271269 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.271410 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.271303 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9bvv\" (UniqueName: \"kubernetes.io/projected/d80d6fd7-1bcc-4672-87d8-78c515faa376-kube-api-access-b9bvv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.271410 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.271376 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.371717 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.371692 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.371805 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.371767 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.371805 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.371791 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b9bvv\" (UniqueName: \"kubernetes.io/projected/d80d6fd7-1bcc-4672-87d8-78c515faa376-kube-api-access-b9bvv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.372065 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.372046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.372110 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.372073 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.381158 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.381136 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9bvv\" (UniqueName: \"kubernetes.io/projected/d80d6fd7-1bcc-4672-87d8-78c515faa376-kube-api-access-b9bvv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.483073 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.483025 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:49.581302 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.581270 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d"] Jun 03 16:27:49.586105 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.586085 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.594011 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.593988 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d"] Jun 03 16:27:49.612543 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:27:49.612521 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd80d6fd7_1bcc_4672_87d8_78c515faa376.slice/crio-3fbd93017d5eda78b9955cfc37868c286c609fa2bcc23adb54b2ecd8c03aabc0 WatchSource:0}: Error finding container 3fbd93017d5eda78b9955cfc37868c286c609fa2bcc23adb54b2ecd8c03aabc0: Status 404 returned error can't find the container with id 3fbd93017d5eda78b9955cfc37868c286c609fa2bcc23adb54b2ecd8c03aabc0 Jun 03 16:27:49.612999 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.612982 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5"] Jun 03 16:27:49.674847 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.674828 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.674957 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.674876 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.674957 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.674924 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j2dc\" (UniqueName: \"kubernetes.io/projected/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-kube-api-access-5j2dc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.755337 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.755297 2576 generic.go:358] "Generic (PLEG): container finished" podID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerID="34c143605dc14f5dcfeb416228bdcf9e8e15065c692fa851e27bafdc2119cd1c" exitCode=0 Jun 03 16:27:49.755432 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.755358 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" event={"ID":"d80d6fd7-1bcc-4672-87d8-78c515faa376","Type":"ContainerDied","Data":"34c143605dc14f5dcfeb416228bdcf9e8e15065c692fa851e27bafdc2119cd1c"} Jun 03 16:27:49.755432 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.755389 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" event={"ID":"d80d6fd7-1bcc-4672-87d8-78c515faa376","Type":"ContainerStarted","Data":"3fbd93017d5eda78b9955cfc37868c286c609fa2bcc23adb54b2ecd8c03aabc0"} Jun 03 16:27:49.756908 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.756888 2576 generic.go:358] "Generic (PLEG): container finished" podID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerID="80fbb210352eb0b043d93cf3cee5bf5c290e9ffffc71f2aa2f033b7d8d40678d" exitCode=0 Jun 03 16:27:49.757008 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.756939 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" event={"ID":"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595","Type":"ContainerDied","Data":"80fbb210352eb0b043d93cf3cee5bf5c290e9ffffc71f2aa2f033b7d8d40678d"} Jun 03 16:27:49.757008 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.756970 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" event={"ID":"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595","Type":"ContainerStarted","Data":"0ef52ea4ce0c2f659b381811412c00023cc902fbdcc757bd7f64fd4f44c1ed5b"} Jun 03 16:27:49.775635 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.775612 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.775731 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.775657 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.775804 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.775764 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5j2dc\" (UniqueName: \"kubernetes.io/projected/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-kube-api-access-5j2dc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.775993 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.775974 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.776050 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.776012 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.784860 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.784833 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j2dc\" (UniqueName: \"kubernetes.io/projected/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-kube-api-access-5j2dc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.898641 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.898619 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:49.976606 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.976566 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd"] Jun 03 16:27:49.981750 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.981727 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:49.987975 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:49.987951 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd"] Jun 03 16:27:50.028374 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.028351 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d"] Jun 03 16:27:50.078668 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.078647 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.078754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.078683 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.078796 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.078767 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt2zj\" (UniqueName: \"kubernetes.io/projected/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-kube-api-access-wt2zj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.179746 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.179718 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.179854 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.179799 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wt2zj\" (UniqueName: \"kubernetes.io/projected/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-kube-api-access-wt2zj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.179977 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.179956 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.180119 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.180099 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.180247 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.180233 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.189176 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.189154 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt2zj\" (UniqueName: \"kubernetes.io/projected/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-kube-api-access-wt2zj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.293941 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.293883 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:50.588369 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.588304 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd"] Jun 03 16:27:50.682072 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:27:50.682048 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3526a40a_477a_45f8_b025_a0a2fcf3ce3e.slice/crio-ebf10912d6e71301c2ed190546bdd1118b0c6ebc9838408df18dab4bf1b69973 WatchSource:0}: Error finding container ebf10912d6e71301c2ed190546bdd1118b0c6ebc9838408df18dab4bf1b69973: Status 404 returned error can't find the container with id ebf10912d6e71301c2ed190546bdd1118b0c6ebc9838408df18dab4bf1b69973 Jun 03 16:27:50.769087 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.769059 2576 generic.go:358] "Generic (PLEG): container finished" podID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerID="a94912e5a1523aad7f0c5bba2d464fd834ae0c2585228d37f27bc8dd0720736a" exitCode=0 Jun 03 16:27:50.769246 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.769139 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" event={"ID":"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595","Type":"ContainerDied","Data":"a94912e5a1523aad7f0c5bba2d464fd834ae0c2585228d37f27bc8dd0720736a"} Jun 03 16:27:50.770685 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.770659 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" event={"ID":"3526a40a-477a-45f8-b025-a0a2fcf3ce3e","Type":"ContainerStarted","Data":"5fa98b80ff58df3dad8f2ef7532703b85c2072279f393836b9a9a37e2a592442"} Jun 03 16:27:50.770776 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.770693 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" event={"ID":"3526a40a-477a-45f8-b025-a0a2fcf3ce3e","Type":"ContainerStarted","Data":"ebf10912d6e71301c2ed190546bdd1118b0c6ebc9838408df18dab4bf1b69973"} Jun 03 16:27:50.772031 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.772010 2576 generic.go:358] "Generic (PLEG): container finished" podID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerID="e456c465daa2646ecd07960bf3b4a779305d10b48a7bad471e737ff985fbc27a" exitCode=0 Jun 03 16:27:50.772146 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.772061 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" event={"ID":"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d","Type":"ContainerDied","Data":"e456c465daa2646ecd07960bf3b4a779305d10b48a7bad471e737ff985fbc27a"} Jun 03 16:27:50.772146 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.772087 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" event={"ID":"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d","Type":"ContainerStarted","Data":"8ea2d8b6313bcb5ae94e7f8162212c408f8efc851bb8de7f49c5d329122a4acc"} Jun 03 16:27:50.773649 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.773626 2576 generic.go:358] "Generic (PLEG): container finished" podID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerID="410d9724988b165b3e3e0c21e477d6b9fdca2fbc2560c0554707d8993ec0eaaf" exitCode=0 Jun 03 16:27:50.773763 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:50.773708 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" event={"ID":"d80d6fd7-1bcc-4672-87d8-78c515faa376","Type":"ContainerDied","Data":"410d9724988b165b3e3e0c21e477d6b9fdca2fbc2560c0554707d8993ec0eaaf"} Jun 03 16:27:51.782724 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:51.782692 2576 generic.go:358] "Generic (PLEG): container finished" podID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerID="5fa98b80ff58df3dad8f2ef7532703b85c2072279f393836b9a9a37e2a592442" exitCode=0 Jun 03 16:27:51.783091 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:51.782771 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" event={"ID":"3526a40a-477a-45f8-b025-a0a2fcf3ce3e","Type":"ContainerDied","Data":"5fa98b80ff58df3dad8f2ef7532703b85c2072279f393836b9a9a37e2a592442"} Jun 03 16:27:51.784472 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:51.784450 2576 generic.go:358] "Generic (PLEG): container finished" podID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerID="6e967c91bb3d903c290fad4417dbea45b889a2a150f1b80863aa3c07b93a2dad" exitCode=0 Jun 03 16:27:51.784553 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:51.784510 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" event={"ID":"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d","Type":"ContainerDied","Data":"6e967c91bb3d903c290fad4417dbea45b889a2a150f1b80863aa3c07b93a2dad"} Jun 03 16:27:51.786426 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:51.786400 2576 generic.go:358] "Generic (PLEG): container finished" podID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerID="c638fefe2cd7388ed6b46a043699458396c17faeab2e1526791c06dbcb9d6335" exitCode=0 Jun 03 16:27:51.786534 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:51.786512 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" event={"ID":"d80d6fd7-1bcc-4672-87d8-78c515faa376","Type":"ContainerDied","Data":"c638fefe2cd7388ed6b46a043699458396c17faeab2e1526791c06dbcb9d6335"} Jun 03 16:27:51.788244 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:51.788226 2576 generic.go:358] "Generic (PLEG): container finished" podID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerID="25dff49e8e09db85ebdc403ff60d37424c12c2c17d4dfaa02a7f3ba9c13d279f" exitCode=0 Jun 03 16:27:51.788340 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:51.788250 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" event={"ID":"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595","Type":"ContainerDied","Data":"25dff49e8e09db85ebdc403ff60d37424c12c2c17d4dfaa02a7f3ba9c13d279f"} Jun 03 16:27:52.793284 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:52.793260 2576 generic.go:358] "Generic (PLEG): container finished" podID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerID="4e5e0b4f2ed182d6c9d653991cd8807faf4e6a268c648d4d2911c167e540dad8" exitCode=0 Jun 03 16:27:52.793559 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:52.793355 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" event={"ID":"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d","Type":"ContainerDied","Data":"4e5e0b4f2ed182d6c9d653991cd8807faf4e6a268c648d4d2911c167e540dad8"} Jun 03 16:27:52.794753 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:52.794731 2576 generic.go:358] "Generic (PLEG): container finished" podID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerID="784d79321d6b62889f5b3d5642eb27d12b262e520fa359c99b700b0b474e1d35" exitCode=0 Jun 03 16:27:52.794831 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:52.794808 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" event={"ID":"3526a40a-477a-45f8-b025-a0a2fcf3ce3e","Type":"ContainerDied","Data":"784d79321d6b62889f5b3d5642eb27d12b262e520fa359c99b700b0b474e1d35"} Jun 03 16:27:52.947080 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:52.947021 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:52.966457 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:52.966428 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:53.102457 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.102429 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-util\") pod \"d80d6fd7-1bcc-4672-87d8-78c515faa376\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " Jun 03 16:27:53.102618 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.102485 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-util\") pod \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " Jun 03 16:27:53.102618 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.102548 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-bundle\") pod \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " Jun 03 16:27:53.102618 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.102576 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cj7r\" (UniqueName: \"kubernetes.io/projected/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-kube-api-access-4cj7r\") pod \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\" (UID: \"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595\") " Jun 03 16:27:53.102618 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.102599 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9bvv\" (UniqueName: \"kubernetes.io/projected/d80d6fd7-1bcc-4672-87d8-78c515faa376-kube-api-access-b9bvv\") pod \"d80d6fd7-1bcc-4672-87d8-78c515faa376\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " Jun 03 16:27:53.102806 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.102633 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-bundle\") pod \"d80d6fd7-1bcc-4672-87d8-78c515faa376\" (UID: \"d80d6fd7-1bcc-4672-87d8-78c515faa376\") " Jun 03 16:27:53.103127 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.103095 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-bundle" (OuterVolumeSpecName: "bundle") pod "d80d6fd7-1bcc-4672-87d8-78c515faa376" (UID: "d80d6fd7-1bcc-4672-87d8-78c515faa376"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:27:53.103261 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.103239 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-bundle" (OuterVolumeSpecName: "bundle") pod "ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" (UID: "ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:27:53.105152 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.105126 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d80d6fd7-1bcc-4672-87d8-78c515faa376-kube-api-access-b9bvv" (OuterVolumeSpecName: "kube-api-access-b9bvv") pod "d80d6fd7-1bcc-4672-87d8-78c515faa376" (UID: "d80d6fd7-1bcc-4672-87d8-78c515faa376"). InnerVolumeSpecName "kube-api-access-b9bvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:27:53.105277 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.105249 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-kube-api-access-4cj7r" (OuterVolumeSpecName: "kube-api-access-4cj7r") pod "ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" (UID: "ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595"). InnerVolumeSpecName "kube-api-access-4cj7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:27:53.107861 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.107826 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-util" (OuterVolumeSpecName: "util") pod "ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" (UID: "ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:27:53.108023 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.108006 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-util" (OuterVolumeSpecName: "util") pod "d80d6fd7-1bcc-4672-87d8-78c515faa376" (UID: "d80d6fd7-1bcc-4672-87d8-78c515faa376"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:27:53.203516 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.203465 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:53.203516 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.203485 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cj7r\" (UniqueName: \"kubernetes.io/projected/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-kube-api-access-4cj7r\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:53.203516 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.203495 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9bvv\" (UniqueName: \"kubernetes.io/projected/d80d6fd7-1bcc-4672-87d8-78c515faa376-kube-api-access-b9bvv\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:53.203516 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.203504 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:53.203516 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.203512 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80d6fd7-1bcc-4672-87d8-78c515faa376-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:53.203516 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.203519 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:53.799932 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.799911 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" Jun 03 16:27:53.800295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.799898 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc" event={"ID":"ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595","Type":"ContainerDied","Data":"0ef52ea4ce0c2f659b381811412c00023cc902fbdcc757bd7f64fd4f44c1ed5b"} Jun 03 16:27:53.800295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.800019 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ef52ea4ce0c2f659b381811412c00023cc902fbdcc757bd7f64fd4f44c1ed5b" Jun 03 16:27:53.801724 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.801699 2576 generic.go:358] "Generic (PLEG): container finished" podID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerID="ff99efcec738c6106ca38cc7399387d53ce9335708e06e4dbef5d5bccbbbb081" exitCode=0 Jun 03 16:27:53.801836 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.801786 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" event={"ID":"3526a40a-477a-45f8-b025-a0a2fcf3ce3e","Type":"ContainerDied","Data":"ff99efcec738c6106ca38cc7399387d53ce9335708e06e4dbef5d5bccbbbb081"} Jun 03 16:27:53.803409 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.803388 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" event={"ID":"d80d6fd7-1bcc-4672-87d8-78c515faa376","Type":"ContainerDied","Data":"3fbd93017d5eda78b9955cfc37868c286c609fa2bcc23adb54b2ecd8c03aabc0"} Jun 03 16:27:53.803499 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.803412 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fbd93017d5eda78b9955cfc37868c286c609fa2bcc23adb54b2ecd8c03aabc0" Jun 03 16:27:53.803499 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.803434 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5" Jun 03 16:27:53.927410 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:53.927391 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:54.009954 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.009929 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-util\") pod \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " Jun 03 16:27:54.010066 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.009986 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j2dc\" (UniqueName: \"kubernetes.io/projected/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-kube-api-access-5j2dc\") pod \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " Jun 03 16:27:54.010066 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.010043 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-bundle\") pod \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\" (UID: \"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d\") " Jun 03 16:27:54.010516 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.010491 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-bundle" (OuterVolumeSpecName: "bundle") pod "134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" (UID: "134b7fe6-30bc-41c7-9bc5-78d1bf07c12d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:27:54.011964 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.011945 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-kube-api-access-5j2dc" (OuterVolumeSpecName: "kube-api-access-5j2dc") pod "134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" (UID: "134b7fe6-30bc-41c7-9bc5-78d1bf07c12d"). InnerVolumeSpecName "kube-api-access-5j2dc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:27:54.017683 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.017642 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-util" (OuterVolumeSpecName: "util") pod "134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" (UID: "134b7fe6-30bc-41c7-9bc5-78d1bf07c12d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:27:54.111087 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.111034 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j2dc\" (UniqueName: \"kubernetes.io/projected/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-kube-api-access-5j2dc\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:54.111087 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.111057 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:54.111087 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.111067 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/134b7fe6-30bc-41c7-9bc5-78d1bf07c12d-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:54.808868 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.808840 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" Jun 03 16:27:54.808868 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.808839 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d" event={"ID":"134b7fe6-30bc-41c7-9bc5-78d1bf07c12d","Type":"ContainerDied","Data":"8ea2d8b6313bcb5ae94e7f8162212c408f8efc851bb8de7f49c5d329122a4acc"} Jun 03 16:27:54.809282 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.808889 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ea2d8b6313bcb5ae94e7f8162212c408f8efc851bb8de7f49c5d329122a4acc" Jun 03 16:27:54.929623 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:54.929603 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:55.017686 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.017661 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-util\") pod \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " Jun 03 16:27:55.017796 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.017697 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-bundle\") pod \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " Jun 03 16:27:55.017796 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.017728 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt2zj\" (UniqueName: \"kubernetes.io/projected/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-kube-api-access-wt2zj\") pod \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\" (UID: \"3526a40a-477a-45f8-b025-a0a2fcf3ce3e\") " Jun 03 16:27:55.018158 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.018135 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-bundle" (OuterVolumeSpecName: "bundle") pod "3526a40a-477a-45f8-b025-a0a2fcf3ce3e" (UID: "3526a40a-477a-45f8-b025-a0a2fcf3ce3e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:27:55.019738 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.019714 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-kube-api-access-wt2zj" (OuterVolumeSpecName: "kube-api-access-wt2zj") pod "3526a40a-477a-45f8-b025-a0a2fcf3ce3e" (UID: "3526a40a-477a-45f8-b025-a0a2fcf3ce3e"). InnerVolumeSpecName "kube-api-access-wt2zj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:27:55.023265 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.023234 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-util" (OuterVolumeSpecName: "util") pod "3526a40a-477a-45f8-b025-a0a2fcf3ce3e" (UID: "3526a40a-477a-45f8-b025-a0a2fcf3ce3e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:27:55.118452 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.118392 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:55.118452 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.118415 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:55.118452 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.118424 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt2zj\" (UniqueName: \"kubernetes.io/projected/3526a40a-477a-45f8-b025-a0a2fcf3ce3e-kube-api-access-wt2zj\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:27:55.814294 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.814267 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" Jun 03 16:27:55.814670 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.814267 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd" event={"ID":"3526a40a-477a-45f8-b025-a0a2fcf3ce3e","Type":"ContainerDied","Data":"ebf10912d6e71301c2ed190546bdd1118b0c6ebc9838408df18dab4bf1b69973"} Jun 03 16:27:55.814670 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:27:55.814360 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebf10912d6e71301c2ed190546bdd1118b0c6ebc9838408df18dab4bf1b69973" Jun 03 16:28:05.705854 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.705820 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-rgz8v"] Jun 03 16:28:05.706344 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706301 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerName="util" Jun 03 16:28:05.706430 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706348 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerName="util" Jun 03 16:28:05.706430 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706372 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerName="pull" Jun 03 16:28:05.706430 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706380 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerName="pull" Jun 03 16:28:05.706430 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706409 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerName="util" Jun 03 16:28:05.706430 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706419 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerName="util" Jun 03 16:28:05.706430 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706428 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerName="util" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706436 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerName="util" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706447 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerName="extract" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706455 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerName="extract" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706473 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerName="util" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706481 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerName="util" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706490 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerName="extract" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706494 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerName="extract" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706501 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerName="pull" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706507 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerName="pull" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706515 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerName="extract" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706523 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerName="extract" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706537 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerName="extract" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706546 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerName="extract" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706557 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerName="pull" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706565 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerName="pull" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706574 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerName="pull" Jun 03 16:28:05.706667 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706581 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerName="pull" Jun 03 16:28:05.707151 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706681 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="134b7fe6-30bc-41c7-9bc5-78d1bf07c12d" containerName="extract" Jun 03 16:28:05.707151 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706705 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="d80d6fd7-1bcc-4672-87d8-78c515faa376" containerName="extract" Jun 03 16:28:05.707151 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706719 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="3526a40a-477a-45f8-b025-a0a2fcf3ce3e" containerName="extract" Jun 03 16:28:05.707151 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.706732 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595" containerName="extract" Jun 03 16:28:05.711221 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.711204 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" Jun 03 16:28:05.714608 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.714590 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5l2wn\"" Jun 03 16:28:05.715592 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.715575 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 16:28:05.715665 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.715590 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 16:28:05.725087 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.725064 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-rgz8v"] Jun 03 16:28:05.905926 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:05.905895 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h7ts\" (UniqueName: \"kubernetes.io/projected/b77ec22e-1b68-426a-a118-746866a3bf8c-kube-api-access-6h7ts\") pod \"authorino-operator-657f44b778-rgz8v\" (UID: \"b77ec22e-1b68-426a-a118-746866a3bf8c\") " pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" Jun 03 16:28:06.007270 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:06.007230 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6h7ts\" (UniqueName: \"kubernetes.io/projected/b77ec22e-1b68-426a-a118-746866a3bf8c-kube-api-access-6h7ts\") pod \"authorino-operator-657f44b778-rgz8v\" (UID: \"b77ec22e-1b68-426a-a118-746866a3bf8c\") " pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" Jun 03 16:28:06.025370 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:06.025343 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h7ts\" (UniqueName: \"kubernetes.io/projected/b77ec22e-1b68-426a-a118-746866a3bf8c-kube-api-access-6h7ts\") pod \"authorino-operator-657f44b778-rgz8v\" (UID: \"b77ec22e-1b68-426a-a118-746866a3bf8c\") " pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" Jun 03 16:28:06.320961 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:06.320888 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" Jun 03 16:28:06.458692 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:06.458664 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-rgz8v"] Jun 03 16:28:06.461749 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:28:06.461726 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77ec22e_1b68_426a_a118_746866a3bf8c.slice/crio-13f747965213115e3e140d0672e8d0f3c1dfb427f36d62c6ac47dbca481c6570 WatchSource:0}: Error finding container 13f747965213115e3e140d0672e8d0f3c1dfb427f36d62c6ac47dbca481c6570: Status 404 returned error can't find the container with id 13f747965213115e3e140d0672e8d0f3c1dfb427f36d62c6ac47dbca481c6570 Jun 03 16:28:06.856858 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:06.856832 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" event={"ID":"b77ec22e-1b68-426a-a118-746866a3bf8c","Type":"ContainerStarted","Data":"13f747965213115e3e140d0672e8d0f3c1dfb427f36d62c6ac47dbca481c6570"} Jun 03 16:28:09.869463 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:09.869427 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" event={"ID":"b77ec22e-1b68-426a-a118-746866a3bf8c","Type":"ContainerStarted","Data":"b125a106ca9213f85646d8fab4b68a15d51957d39bbb596481f444d9ecdd5df3"} Jun 03 16:28:09.869899 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:09.869485 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" Jun 03 16:28:09.908658 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:09.908609 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" podStartSLOduration=2.583143432 podStartE2EDuration="4.908595587s" podCreationTimestamp="2026-06-03 16:28:05 +0000 UTC" firstStartedPulling="2026-06-03 16:28:06.463522266 +0000 UTC m=+416.727586122" lastFinishedPulling="2026-06-03 16:28:08.788974423 +0000 UTC m=+419.053038277" observedRunningTime="2026-06-03 16:28:09.903775384 +0000 UTC m=+420.167839251" watchObservedRunningTime="2026-06-03 16:28:09.908595587 +0000 UTC m=+420.172659461" Jun 03 16:28:17.386577 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.386543 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9"] Jun 03 16:28:17.391567 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.391542 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:17.396422 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.396400 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qf2wj\"" Jun 03 16:28:17.408068 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.408044 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9"] Jun 03 16:28:17.500656 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.500632 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb88k\" (UniqueName: \"kubernetes.io/projected/f4a15194-f5f5-484b-962e-864c5a8af073-kube-api-access-fb88k\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" (UID: \"f4a15194-f5f5-484b-962e-864c5a8af073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:17.500766 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.500692 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4a15194-f5f5-484b-962e-864c5a8af073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" (UID: \"f4a15194-f5f5-484b-962e-864c5a8af073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:17.601011 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.600974 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4a15194-f5f5-484b-962e-864c5a8af073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" (UID: \"f4a15194-f5f5-484b-962e-864c5a8af073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:17.601125 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.601035 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fb88k\" (UniqueName: \"kubernetes.io/projected/f4a15194-f5f5-484b-962e-864c5a8af073-kube-api-access-fb88k\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" (UID: \"f4a15194-f5f5-484b-962e-864c5a8af073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:17.601364 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.601348 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4a15194-f5f5-484b-962e-864c5a8af073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" (UID: \"f4a15194-f5f5-484b-962e-864c5a8af073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:17.609990 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.609968 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb88k\" (UniqueName: \"kubernetes.io/projected/f4a15194-f5f5-484b-962e-864c5a8af073-kube-api-access-fb88k\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" (UID: \"f4a15194-f5f5-484b-962e-864c5a8af073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:17.702209 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.702144 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:17.832985 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.832960 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9"] Jun 03 16:28:17.833530 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:28:17.833501 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a15194_f5f5_484b_962e_864c5a8af073.slice/crio-da9ca279cef04b85ff396a6b898f34f4c36e3252263b1cc2ae661e057bf21440 WatchSource:0}: Error finding container da9ca279cef04b85ff396a6b898f34f4c36e3252263b1cc2ae661e057bf21440: Status 404 returned error can't find the container with id da9ca279cef04b85ff396a6b898f34f4c36e3252263b1cc2ae661e057bf21440 Jun 03 16:28:17.902692 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:17.902670 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" event={"ID":"f4a15194-f5f5-484b-962e-864c5a8af073","Type":"ContainerStarted","Data":"da9ca279cef04b85ff396a6b898f34f4c36e3252263b1cc2ae661e057bf21440"} Jun 03 16:28:20.875412 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:20.875378 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-rgz8v" Jun 03 16:28:24.931198 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:24.931161 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" event={"ID":"f4a15194-f5f5-484b-962e-864c5a8af073","Type":"ContainerStarted","Data":"251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae"} Jun 03 16:28:24.931563 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:24.931306 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:24.955253 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:24.955209 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" podStartSLOduration=1.526267786 podStartE2EDuration="7.955195366s" podCreationTimestamp="2026-06-03 16:28:17 +0000 UTC" firstStartedPulling="2026-06-03 16:28:17.836258219 +0000 UTC m=+428.100322075" lastFinishedPulling="2026-06-03 16:28:24.2651858 +0000 UTC m=+434.529249655" observedRunningTime="2026-06-03 16:28:24.952025971 +0000 UTC m=+435.216089847" watchObservedRunningTime="2026-06-03 16:28:24.955195366 +0000 UTC m=+435.219259241" Jun 03 16:28:35.936877 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:35.936850 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:37.616768 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.616739 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9"] Jun 03 16:28:37.617132 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.616965 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" containerName="manager" containerID="cri-o://251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae" gracePeriod=2 Jun 03 16:28:37.624418 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.624389 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9"] Jun 03 16:28:37.682370 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.682345 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc"] Jun 03 16:28:37.682706 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.682694 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" containerName="manager" Jun 03 16:28:37.682748 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.682708 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" containerName="manager" Jun 03 16:28:37.682794 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.682785 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" containerName="manager" Jun 03 16:28:37.685616 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.685600 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:37.689251 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.689226 2576 status_manager.go:895] "Failed to get status for pod" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" Jun 03 16:28:37.702844 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.702824 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc"] Jun 03 16:28:37.762726 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.762683 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10816964-def9-43ff-99bc-7dcbcd2be4ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j2pvc\" (UID: \"10816964-def9-43ff-99bc-7dcbcd2be4ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:37.762882 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.762860 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdb5v\" (UniqueName: \"kubernetes.io/projected/10816964-def9-43ff-99bc-7dcbcd2be4ca-kube-api-access-rdb5v\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j2pvc\" (UID: \"10816964-def9-43ff-99bc-7dcbcd2be4ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:37.858344 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.858310 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:37.861260 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.861228 2576 status_manager.go:895] "Failed to get status for pod" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" Jun 03 16:28:37.863642 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.863624 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rdb5v\" (UniqueName: \"kubernetes.io/projected/10816964-def9-43ff-99bc-7dcbcd2be4ca-kube-api-access-rdb5v\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j2pvc\" (UID: \"10816964-def9-43ff-99bc-7dcbcd2be4ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:37.863708 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.863682 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10816964-def9-43ff-99bc-7dcbcd2be4ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j2pvc\" (UID: \"10816964-def9-43ff-99bc-7dcbcd2be4ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:37.863971 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.863957 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10816964-def9-43ff-99bc-7dcbcd2be4ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j2pvc\" (UID: \"10816964-def9-43ff-99bc-7dcbcd2be4ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:37.872761 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.872706 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdb5v\" (UniqueName: \"kubernetes.io/projected/10816964-def9-43ff-99bc-7dcbcd2be4ca-kube-api-access-rdb5v\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j2pvc\" (UID: \"10816964-def9-43ff-99bc-7dcbcd2be4ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:37.964098 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.964070 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb88k\" (UniqueName: \"kubernetes.io/projected/f4a15194-f5f5-484b-962e-864c5a8af073-kube-api-access-fb88k\") pod \"f4a15194-f5f5-484b-962e-864c5a8af073\" (UID: \"f4a15194-f5f5-484b-962e-864c5a8af073\") " Jun 03 16:28:37.964194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.964167 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4a15194-f5f5-484b-962e-864c5a8af073-extensions-socket-volume\") pod \"f4a15194-f5f5-484b-962e-864c5a8af073\" (UID: \"f4a15194-f5f5-484b-962e-864c5a8af073\") " Jun 03 16:28:37.964626 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.964603 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4a15194-f5f5-484b-962e-864c5a8af073-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f4a15194-f5f5-484b-962e-864c5a8af073" (UID: "f4a15194-f5f5-484b-962e-864c5a8af073"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:28:37.965950 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.965917 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4a15194-f5f5-484b-962e-864c5a8af073-kube-api-access-fb88k" (OuterVolumeSpecName: "kube-api-access-fb88k") pod "f4a15194-f5f5-484b-962e-864c5a8af073" (UID: "f4a15194-f5f5-484b-962e-864c5a8af073"). InnerVolumeSpecName "kube-api-access-fb88k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:28:37.980797 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.980772 2576 generic.go:358] "Generic (PLEG): container finished" podID="f4a15194-f5f5-484b-962e-864c5a8af073" containerID="251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae" exitCode=0 Jun 03 16:28:37.980907 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.980807 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" Jun 03 16:28:37.980907 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.980855 2576 scope.go:117] "RemoveContainer" containerID="251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae" Jun 03 16:28:37.983345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.983296 2576 status_manager.go:895] "Failed to get status for pod" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" Jun 03 16:28:37.994070 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.994001 2576 scope.go:117] "RemoveContainer" containerID="251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae" Jun 03 16:28:37.994446 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:28:37.994419 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae\": container with ID starting with 251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae not found: ID does not exist" containerID="251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae" Jun 03 16:28:37.994545 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.994455 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae"} err="failed to get container status \"251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae\": rpc error: code = NotFound desc = could not find container \"251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae\": container with ID starting with 251ff0b5bdaf0fe598c0b031780e6c4e466577aac8a73a497a11609e846c2bae not found: ID does not exist" Jun 03 16:28:37.994545 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.994473 2576 status_manager.go:895] "Failed to get status for pod" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tvwx9" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tvwx9\" is forbidden: User \"system:node:ip-10-0-128-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-141.ec2.internal' and this object" Jun 03 16:28:37.996405 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:37.996389 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:38.005257 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:28:38.005231 2576 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a15194_f5f5_484b_962e_864c5a8af073.slice\": RecentStats: unable to find data in memory cache]" Jun 03 16:28:38.005352 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:28:38.005332 2576 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a15194_f5f5_484b_962e_864c5a8af073.slice\": RecentStats: unable to find data in memory cache]" Jun 03 16:28:38.065672 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:38.065647 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb88k\" (UniqueName: \"kubernetes.io/projected/f4a15194-f5f5-484b-962e-864c5a8af073-kube-api-access-fb88k\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:28:38.065672 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:38.065671 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4a15194-f5f5-484b-962e-864c5a8af073-extensions-socket-volume\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:28:38.117784 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:38.117759 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc"] Jun 03 16:28:38.118949 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:28:38.118922 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10816964_def9_43ff_99bc_7dcbcd2be4ca.slice/crio-fd66c1e86f6769963a2739f6d45e5a5d08369073f5611374c5df0587ac42a322 WatchSource:0}: Error finding container fd66c1e86f6769963a2739f6d45e5a5d08369073f5611374c5df0587ac42a322: Status 404 returned error can't find the container with id fd66c1e86f6769963a2739f6d45e5a5d08369073f5611374c5df0587ac42a322 Jun 03 16:28:38.314892 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:38.314866 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4a15194-f5f5-484b-962e-864c5a8af073" path="/var/lib/kubelet/pods/f4a15194-f5f5-484b-962e-864c5a8af073/volumes" Jun 03 16:28:38.987066 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:38.987031 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" event={"ID":"10816964-def9-43ff-99bc-7dcbcd2be4ca","Type":"ContainerStarted","Data":"e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8"} Jun 03 16:28:38.987066 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:38.987067 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" event={"ID":"10816964-def9-43ff-99bc-7dcbcd2be4ca","Type":"ContainerStarted","Data":"fd66c1e86f6769963a2739f6d45e5a5d08369073f5611374c5df0587ac42a322"} Jun 03 16:28:38.987491 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:38.987099 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:39.024938 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:39.024893 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" podStartSLOduration=2.024879088 podStartE2EDuration="2.024879088s" podCreationTimestamp="2026-06-03 16:28:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:28:39.021582796 +0000 UTC m=+449.285646671" watchObservedRunningTime="2026-06-03 16:28:39.024879088 +0000 UTC m=+449.288942962" Jun 03 16:28:43.275464 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.275429 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-645b8bb455-8kknv"] Jun 03 16:28:43.280047 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.280024 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.290209 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.290176 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645b8bb455-8kknv"] Jun 03 16:28:43.407276 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.407247 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-oauth-serving-cert\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.407428 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.407294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-trusted-ca-bundle\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.407428 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.407339 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-oauth-config\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.407428 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.407362 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-service-ca\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.407428 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.407387 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-config\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.407428 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.407423 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4t97\" (UniqueName: \"kubernetes.io/projected/5ebfaa11-f736-498d-9573-9d028b5f0f82-kube-api-access-z4t97\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.407638 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.407466 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-serving-cert\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.508560 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.508532 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-trusted-ca-bundle\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.508669 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.508564 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-oauth-config\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.508669 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.508582 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-service-ca\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.508669 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.508604 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-config\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.508820 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.508781 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z4t97\" (UniqueName: \"kubernetes.io/projected/5ebfaa11-f736-498d-9573-9d028b5f0f82-kube-api-access-z4t97\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.508867 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.508837 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-serving-cert\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.508927 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.508911 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-oauth-serving-cert\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.509306 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.509273 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-config\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.509564 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.509543 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-trusted-ca-bundle\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.509639 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.509550 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-service-ca\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.509747 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.509724 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ebfaa11-f736-498d-9573-9d028b5f0f82-oauth-serving-cert\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.511214 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.511192 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-oauth-config\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.511480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.511458 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ebfaa11-f736-498d-9573-9d028b5f0f82-console-serving-cert\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.516637 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.516614 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4t97\" (UniqueName: \"kubernetes.io/projected/5ebfaa11-f736-498d-9573-9d028b5f0f82-kube-api-access-z4t97\") pod \"console-645b8bb455-8kknv\" (UID: \"5ebfaa11-f736-498d-9573-9d028b5f0f82\") " pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.591174 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.591121 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:43.720254 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:43.720229 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645b8bb455-8kknv"] Jun 03 16:28:43.721649 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:28:43.721613 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ebfaa11_f736_498d_9573_9d028b5f0f82.slice/crio-43d39cb348b20003bbf9b27f1556635b5444ef116f7035954b508f25a118a121 WatchSource:0}: Error finding container 43d39cb348b20003bbf9b27f1556635b5444ef116f7035954b508f25a118a121: Status 404 returned error can't find the container with id 43d39cb348b20003bbf9b27f1556635b5444ef116f7035954b508f25a118a121 Jun 03 16:28:44.012811 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:44.012778 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645b8bb455-8kknv" event={"ID":"5ebfaa11-f736-498d-9573-9d028b5f0f82","Type":"ContainerStarted","Data":"575d89a8b85f2b43b8696daa2f5ebfc6dea5e415dbb938471222ae1d010024e8"} Jun 03 16:28:44.012811 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:44.012814 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645b8bb455-8kknv" event={"ID":"5ebfaa11-f736-498d-9573-9d028b5f0f82","Type":"ContainerStarted","Data":"43d39cb348b20003bbf9b27f1556635b5444ef116f7035954b508f25a118a121"} Jun 03 16:28:44.036408 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:44.036364 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-645b8bb455-8kknv" podStartSLOduration=1.036351046 podStartE2EDuration="1.036351046s" podCreationTimestamp="2026-06-03 16:28:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:28:44.033170261 +0000 UTC m=+454.297234135" watchObservedRunningTime="2026-06-03 16:28:44.036351046 +0000 UTC m=+454.300414920" Jun 03 16:28:49.993625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:49.993598 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:28:53.591589 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:53.591554 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:53.592005 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:53.591813 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:53.596112 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:53.596091 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:54.059153 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:54.059126 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-645b8bb455-8kknv" Jun 03 16:28:54.123940 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:28:54.123912 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cb5db9c64-2gl4t"] Jun 03 16:29:07.784834 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:07.784799 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc"] Jun 03 16:29:07.785237 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:07.785027 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" podUID="10816964-def9-43ff-99bc-7dcbcd2be4ca" containerName="manager" containerID="cri-o://e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8" gracePeriod=10 Jun 03 16:29:08.023082 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.023058 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:29:08.036941 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.036887 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx"] Jun 03 16:29:08.037252 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.037241 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10816964-def9-43ff-99bc-7dcbcd2be4ca" containerName="manager" Jun 03 16:29:08.037292 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.037253 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="10816964-def9-43ff-99bc-7dcbcd2be4ca" containerName="manager" Jun 03 16:29:08.037348 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.037339 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="10816964-def9-43ff-99bc-7dcbcd2be4ca" containerName="manager" Jun 03 16:29:08.039214 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.039201 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:08.052071 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.052049 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx"] Jun 03 16:29:08.114045 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.114016 2576 generic.go:358] "Generic (PLEG): container finished" podID="10816964-def9-43ff-99bc-7dcbcd2be4ca" containerID="e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8" exitCode=0 Jun 03 16:29:08.114173 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.114075 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" Jun 03 16:29:08.114173 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.114101 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" event={"ID":"10816964-def9-43ff-99bc-7dcbcd2be4ca","Type":"ContainerDied","Data":"e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8"} Jun 03 16:29:08.114173 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.114143 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc" event={"ID":"10816964-def9-43ff-99bc-7dcbcd2be4ca","Type":"ContainerDied","Data":"fd66c1e86f6769963a2739f6d45e5a5d08369073f5611374c5df0587ac42a322"} Jun 03 16:29:08.114173 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.114160 2576 scope.go:117] "RemoveContainer" containerID="e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8" Jun 03 16:29:08.122223 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.122205 2576 scope.go:117] "RemoveContainer" containerID="e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8" Jun 03 16:29:08.122504 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:29:08.122483 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8\": container with ID starting with e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8 not found: ID does not exist" containerID="e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8" Jun 03 16:29:08.122594 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.122511 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8"} err="failed to get container status \"e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8\": rpc error: code = NotFound desc = could not find container \"e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8\": container with ID starting with e5d705acf169886678bdbca040ee43939711e98defb07085f5eb182d026649f8 not found: ID does not exist" Jun 03 16:29:08.196948 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.196928 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10816964-def9-43ff-99bc-7dcbcd2be4ca-extensions-socket-volume\") pod \"10816964-def9-43ff-99bc-7dcbcd2be4ca\" (UID: \"10816964-def9-43ff-99bc-7dcbcd2be4ca\") " Jun 03 16:29:08.197030 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.196969 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdb5v\" (UniqueName: \"kubernetes.io/projected/10816964-def9-43ff-99bc-7dcbcd2be4ca-kube-api-access-rdb5v\") pod \"10816964-def9-43ff-99bc-7dcbcd2be4ca\" (UID: \"10816964-def9-43ff-99bc-7dcbcd2be4ca\") " Jun 03 16:29:08.197196 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.197177 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnwwg\" (UniqueName: \"kubernetes.io/projected/e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3-kube-api-access-jnwwg\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5kcsx\" (UID: \"e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:08.197347 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.197299 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10816964-def9-43ff-99bc-7dcbcd2be4ca-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "10816964-def9-43ff-99bc-7dcbcd2be4ca" (UID: "10816964-def9-43ff-99bc-7dcbcd2be4ca"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:29:08.197410 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.197370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5kcsx\" (UID: \"e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:08.197453 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.197415 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10816964-def9-43ff-99bc-7dcbcd2be4ca-extensions-socket-volume\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:08.198897 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.198875 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10816964-def9-43ff-99bc-7dcbcd2be4ca-kube-api-access-rdb5v" (OuterVolumeSpecName: "kube-api-access-rdb5v") pod "10816964-def9-43ff-99bc-7dcbcd2be4ca" (UID: "10816964-def9-43ff-99bc-7dcbcd2be4ca"). InnerVolumeSpecName "kube-api-access-rdb5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:29:08.298744 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.298687 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5kcsx\" (UID: \"e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:08.298744 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.298733 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jnwwg\" (UniqueName: \"kubernetes.io/projected/e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3-kube-api-access-jnwwg\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5kcsx\" (UID: \"e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:08.298883 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.298853 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdb5v\" (UniqueName: \"kubernetes.io/projected/10816964-def9-43ff-99bc-7dcbcd2be4ca-kube-api-access-rdb5v\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:08.299078 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.299058 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5kcsx\" (UID: \"e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:08.323634 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.323611 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnwwg\" (UniqueName: \"kubernetes.io/projected/e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3-kube-api-access-jnwwg\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5kcsx\" (UID: \"e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:08.349427 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.349410 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:08.435977 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.435950 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc"] Jun 03 16:29:08.442271 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.442242 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j2pvc"] Jun 03 16:29:08.482569 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:08.482548 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx"] Jun 03 16:29:08.485367 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:29:08.485344 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d3eb2e_e2a8_413d_ab03_2cb5da15dee3.slice/crio-6b577502b7203fdc68f79392e53c5f891964981b511d83c4da664635cafed44a WatchSource:0}: Error finding container 6b577502b7203fdc68f79392e53c5f891964981b511d83c4da664635cafed44a: Status 404 returned error can't find the container with id 6b577502b7203fdc68f79392e53c5f891964981b511d83c4da664635cafed44a Jun 03 16:29:09.119499 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:09.119447 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" event={"ID":"e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3","Type":"ContainerStarted","Data":"bb98fd42177b51ea9e00cd059b7aa7432cf319525a13801197d316ec03627c76"} Jun 03 16:29:09.119499 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:09.119505 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" event={"ID":"e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3","Type":"ContainerStarted","Data":"6b577502b7203fdc68f79392e53c5f891964981b511d83c4da664635cafed44a"} Jun 03 16:29:09.119940 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:09.119599 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:09.146342 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:09.146275 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" podStartSLOduration=1.146259379 podStartE2EDuration="1.146259379s" podCreationTimestamp="2026-06-03 16:29:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:29:09.142720193 +0000 UTC m=+479.406784058" watchObservedRunningTime="2026-06-03 16:29:09.146259379 +0000 UTC m=+479.410323260" Jun 03 16:29:10.314771 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:10.314745 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10816964-def9-43ff-99bc-7dcbcd2be4ca" path="/var/lib/kubelet/pods/10816964-def9-43ff-99bc-7dcbcd2be4ca/volumes" Jun 03 16:29:19.146477 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.146412 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7cb5db9c64-2gl4t" podUID="068e2ddc-c216-4d70-a8ab-397efbead753" containerName="console" containerID="cri-o://db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5" gracePeriod=15 Jun 03 16:29:19.395462 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.395441 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cb5db9c64-2gl4t_068e2ddc-c216-4d70-a8ab-397efbead753/console/0.log" Jun 03 16:29:19.395577 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.395510 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:29:19.578962 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.578935 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-serving-cert\") pod \"068e2ddc-c216-4d70-a8ab-397efbead753\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " Jun 03 16:29:19.579115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.578983 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s4hn\" (UniqueName: \"kubernetes.io/projected/068e2ddc-c216-4d70-a8ab-397efbead753-kube-api-access-5s4hn\") pod \"068e2ddc-c216-4d70-a8ab-397efbead753\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " Jun 03 16:29:19.579115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579017 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-console-config\") pod \"068e2ddc-c216-4d70-a8ab-397efbead753\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " Jun 03 16:29:19.579115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579040 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-oauth-config\") pod \"068e2ddc-c216-4d70-a8ab-397efbead753\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " Jun 03 16:29:19.579115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579063 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-trusted-ca-bundle\") pod \"068e2ddc-c216-4d70-a8ab-397efbead753\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " Jun 03 16:29:19.579115 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579086 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-oauth-serving-cert\") pod \"068e2ddc-c216-4d70-a8ab-397efbead753\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " Jun 03 16:29:19.579384 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579181 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-service-ca\") pod \"068e2ddc-c216-4d70-a8ab-397efbead753\" (UID: \"068e2ddc-c216-4d70-a8ab-397efbead753\") " Jun 03 16:29:19.579533 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579498 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-console-config" (OuterVolumeSpecName: "console-config") pod "068e2ddc-c216-4d70-a8ab-397efbead753" (UID: "068e2ddc-c216-4d70-a8ab-397efbead753"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:29:19.579533 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579529 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "068e2ddc-c216-4d70-a8ab-397efbead753" (UID: "068e2ddc-c216-4d70-a8ab-397efbead753"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:29:19.579710 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579525 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "068e2ddc-c216-4d70-a8ab-397efbead753" (UID: "068e2ddc-c216-4d70-a8ab-397efbead753"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:29:19.579799 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.579777 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-service-ca" (OuterVolumeSpecName: "service-ca") pod "068e2ddc-c216-4d70-a8ab-397efbead753" (UID: "068e2ddc-c216-4d70-a8ab-397efbead753"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:29:19.581359 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.581314 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "068e2ddc-c216-4d70-a8ab-397efbead753" (UID: "068e2ddc-c216-4d70-a8ab-397efbead753"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:29:19.581466 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.581362 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/068e2ddc-c216-4d70-a8ab-397efbead753-kube-api-access-5s4hn" (OuterVolumeSpecName: "kube-api-access-5s4hn") pod "068e2ddc-c216-4d70-a8ab-397efbead753" (UID: "068e2ddc-c216-4d70-a8ab-397efbead753"). InnerVolumeSpecName "kube-api-access-5s4hn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:29:19.581466 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.581405 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "068e2ddc-c216-4d70-a8ab-397efbead753" (UID: "068e2ddc-c216-4d70-a8ab-397efbead753"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:29:19.680255 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.680235 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s4hn\" (UniqueName: \"kubernetes.io/projected/068e2ddc-c216-4d70-a8ab-397efbead753-kube-api-access-5s4hn\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:19.680255 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.680254 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-console-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:19.680405 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.680263 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-oauth-config\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:19.680405 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.680272 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-trusted-ca-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:19.680405 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.680281 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-oauth-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:19.680405 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.680290 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/068e2ddc-c216-4d70-a8ab-397efbead753-service-ca\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:19.680405 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:19.680299 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/068e2ddc-c216-4d70-a8ab-397efbead753-console-serving-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:29:20.126266 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.126243 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5kcsx" Jun 03 16:29:20.165202 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.165184 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cb5db9c64-2gl4t_068e2ddc-c216-4d70-a8ab-397efbead753/console/0.log" Jun 03 16:29:20.165569 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.165218 2576 generic.go:358] "Generic (PLEG): container finished" podID="068e2ddc-c216-4d70-a8ab-397efbead753" containerID="db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5" exitCode=2 Jun 03 16:29:20.165569 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.165256 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb5db9c64-2gl4t" event={"ID":"068e2ddc-c216-4d70-a8ab-397efbead753","Type":"ContainerDied","Data":"db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5"} Jun 03 16:29:20.165569 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.165275 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb5db9c64-2gl4t" event={"ID":"068e2ddc-c216-4d70-a8ab-397efbead753","Type":"ContainerDied","Data":"ef87f857978f24b792b505483fb0877e04f5ee005a5051237347b6311ca85b4e"} Jun 03 16:29:20.165569 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.165292 2576 scope.go:117] "RemoveContainer" containerID="db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5" Jun 03 16:29:20.165569 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.165295 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb5db9c64-2gl4t" Jun 03 16:29:20.176530 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.176514 2576 scope.go:117] "RemoveContainer" containerID="db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5" Jun 03 16:29:20.176812 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:29:20.176794 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5\": container with ID starting with db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5 not found: ID does not exist" containerID="db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5" Jun 03 16:29:20.176883 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.176823 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5"} err="failed to get container status \"db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5\": rpc error: code = NotFound desc = could not find container \"db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5\": container with ID starting with db7a3043b4ffb16f1ad508bb089db1e04a40666e4e7e1d2034ef1d9b0902d3b5 not found: ID does not exist" Jun 03 16:29:20.200521 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.200499 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cb5db9c64-2gl4t"] Jun 03 16:29:20.207476 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.207448 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cb5db9c64-2gl4t"] Jun 03 16:29:20.316572 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:20.316545 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="068e2ddc-c216-4d70-a8ab-397efbead753" path="/var/lib/kubelet/pods/068e2ddc-c216-4d70-a8ab-397efbead753/volumes" Jun 03 16:29:29.322346 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.322285 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-bfb4n"] Jun 03 16:29:29.322847 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.322824 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="068e2ddc-c216-4d70-a8ab-397efbead753" containerName="console" Jun 03 16:29:29.322847 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.322841 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="068e2ddc-c216-4d70-a8ab-397efbead753" containerName="console" Jun 03 16:29:29.322934 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.322911 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="068e2ddc-c216-4d70-a8ab-397efbead753" containerName="console" Jun 03 16:29:29.327256 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.327239 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-bfb4n" Jun 03 16:29:29.329688 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.329672 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8c72d\"" Jun 03 16:29:29.335166 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.335144 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-bfb4n"] Jun 03 16:29:29.346621 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.346593 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbt22\" (UniqueName: \"kubernetes.io/projected/adbad3a8-1830-443d-8258-fe09018aaeca-kube-api-access-mbt22\") pod \"authorino-7498df8756-bfb4n\" (UID: \"adbad3a8-1830-443d-8258-fe09018aaeca\") " pod="kuadrant-system/authorino-7498df8756-bfb4n" Jun 03 16:29:29.447094 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.447046 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mbt22\" (UniqueName: \"kubernetes.io/projected/adbad3a8-1830-443d-8258-fe09018aaeca-kube-api-access-mbt22\") pod \"authorino-7498df8756-bfb4n\" (UID: \"adbad3a8-1830-443d-8258-fe09018aaeca\") " pod="kuadrant-system/authorino-7498df8756-bfb4n" Jun 03 16:29:29.456871 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.456842 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbt22\" (UniqueName: \"kubernetes.io/projected/adbad3a8-1830-443d-8258-fe09018aaeca-kube-api-access-mbt22\") pod \"authorino-7498df8756-bfb4n\" (UID: \"adbad3a8-1830-443d-8258-fe09018aaeca\") " pod="kuadrant-system/authorino-7498df8756-bfb4n" Jun 03 16:29:29.637803 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.637730 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-bfb4n" Jun 03 16:29:29.759625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:29.759598 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-bfb4n"] Jun 03 16:29:29.761125 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:29:29.761096 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadbad3a8_1830_443d_8258_fe09018aaeca.slice/crio-f79e85146e8c034427381a8f2bf0281ebb8d35f194a1b80fa91f37c8a665ce62 WatchSource:0}: Error finding container f79e85146e8c034427381a8f2bf0281ebb8d35f194a1b80fa91f37c8a665ce62: Status 404 returned error can't find the container with id f79e85146e8c034427381a8f2bf0281ebb8d35f194a1b80fa91f37c8a665ce62 Jun 03 16:29:30.204194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:30.204157 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-bfb4n" event={"ID":"adbad3a8-1830-443d-8258-fe09018aaeca","Type":"ContainerStarted","Data":"f79e85146e8c034427381a8f2bf0281ebb8d35f194a1b80fa91f37c8a665ce62"} Jun 03 16:29:33.218931 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:33.218893 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-bfb4n" event={"ID":"adbad3a8-1830-443d-8258-fe09018aaeca","Type":"ContainerStarted","Data":"246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6"} Jun 03 16:29:33.242342 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:29:33.242285 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-bfb4n" podStartSLOduration=1.3573030369999999 podStartE2EDuration="4.242273462s" podCreationTimestamp="2026-06-03 16:29:29 +0000 UTC" firstStartedPulling="2026-06-03 16:29:29.762729657 +0000 UTC m=+500.026793513" lastFinishedPulling="2026-06-03 16:29:32.647700081 +0000 UTC m=+502.911763938" observedRunningTime="2026-06-03 16:29:33.240930903 +0000 UTC m=+503.504994776" watchObservedRunningTime="2026-06-03 16:29:33.242273462 +0000 UTC m=+503.506337381" Jun 03 16:30:00.726713 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.726642 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2"] Jun 03 16:30:00.730387 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.730369 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.733268 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.733250 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:30:00.733268 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.733259 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:30:00.734232 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.734219 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p5n4l\"" Jun 03 16:30:00.742018 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.741991 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2"] Jun 03 16:30:00.796454 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.796429 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92qcn\" (UniqueName: \"kubernetes.io/projected/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-kube-api-access-92qcn\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.796552 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.796473 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.796552 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.796508 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.897733 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.897707 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.897823 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.897755 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-92qcn\" (UniqueName: \"kubernetes.io/projected/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-kube-api-access-92qcn\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.897823 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.897789 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.898165 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.898149 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.898201 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.898155 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:00.908923 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:00.908898 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92qcn\" (UniqueName: \"kubernetes.io/projected/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-kube-api-access-92qcn\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:01.039519 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:01.039498 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:01.171614 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:01.171588 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2"] Jun 03 16:30:01.173356 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:30:01.173316 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d9dad3_b93e_40c4_abb8_5aa42d0f8af9.slice/crio-103f389ef6c24f8a711aac19fd083dc4c3995e43c3dbad6a05de6f2221d463fe WatchSource:0}: Error finding container 103f389ef6c24f8a711aac19fd083dc4c3995e43c3dbad6a05de6f2221d463fe: Status 404 returned error can't find the container with id 103f389ef6c24f8a711aac19fd083dc4c3995e43c3dbad6a05de6f2221d463fe Jun 03 16:30:01.338295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:01.338232 2576 generic.go:358] "Generic (PLEG): container finished" podID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerID="b6cd004332baf290cd3c48d422da3e34cc33bad9e3118da1dfdb1048fa78d0a1" exitCode=0 Jun 03 16:30:01.338295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:01.338284 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" event={"ID":"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9","Type":"ContainerDied","Data":"b6cd004332baf290cd3c48d422da3e34cc33bad9e3118da1dfdb1048fa78d0a1"} Jun 03 16:30:01.338437 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:01.338302 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" event={"ID":"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9","Type":"ContainerStarted","Data":"103f389ef6c24f8a711aac19fd083dc4c3995e43c3dbad6a05de6f2221d463fe"} Jun 03 16:30:02.343036 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:02.343001 2576 generic.go:358] "Generic (PLEG): container finished" podID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerID="e6149d9bb5f36fac01e1cb6b80c07463498ea770ae08235619566a63a08c099e" exitCode=0 Jun 03 16:30:02.343438 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:02.343089 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" event={"ID":"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9","Type":"ContainerDied","Data":"e6149d9bb5f36fac01e1cb6b80c07463498ea770ae08235619566a63a08c099e"} Jun 03 16:30:03.348493 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:03.348462 2576 generic.go:358] "Generic (PLEG): container finished" podID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerID="ca489baee77304c05c3c2442823b217ed47c6ec957df2410ad416219e0127f74" exitCode=0 Jun 03 16:30:03.348866 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:03.348549 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" event={"ID":"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9","Type":"ContainerDied","Data":"ca489baee77304c05c3c2442823b217ed47c6ec957df2410ad416219e0127f74"} Jun 03 16:30:04.481413 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.481391 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:04.629666 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.629586 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-bundle\") pod \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " Jun 03 16:30:04.629793 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.629713 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92qcn\" (UniqueName: \"kubernetes.io/projected/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-kube-api-access-92qcn\") pod \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " Jun 03 16:30:04.629793 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.629737 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-util\") pod \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\" (UID: \"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9\") " Jun 03 16:30:04.630207 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.630175 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-bundle" (OuterVolumeSpecName: "bundle") pod "a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" (UID: "a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:30:04.631728 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.631703 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-kube-api-access-92qcn" (OuterVolumeSpecName: "kube-api-access-92qcn") pod "a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" (UID: "a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9"). InnerVolumeSpecName "kube-api-access-92qcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:30:04.635204 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.635180 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-util" (OuterVolumeSpecName: "util") pod "a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" (UID: "a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:30:04.730479 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.730435 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92qcn\" (UniqueName: \"kubernetes.io/projected/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-kube-api-access-92qcn\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:30:04.730479 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.730475 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-util\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:30:04.730479 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:04.730485 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9-bundle\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:30:05.357820 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:05.357791 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" Jun 03 16:30:05.357978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:05.357817 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70bcnx2" event={"ID":"a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9","Type":"ContainerDied","Data":"103f389ef6c24f8a711aac19fd083dc4c3995e43c3dbad6a05de6f2221d463fe"} Jun 03 16:30:05.357978 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:30:05.357843 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="103f389ef6c24f8a711aac19fd083dc4c3995e43c3dbad6a05de6f2221d463fe" Jun 03 16:31:02.652287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.649410 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-674dd879f4-nqz52"] Jun 03 16:31:02.652287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.650271 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerName="pull" Jun 03 16:31:02.652287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.650292 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerName="pull" Jun 03 16:31:02.652287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.650302 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerName="extract" Jun 03 16:31:02.652287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.650310 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerName="extract" Jun 03 16:31:02.652287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.650370 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerName="util" Jun 03 16:31:02.652287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.650379 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerName="util" Jun 03 16:31:02.652287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.650558 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="a8d9dad3-b93e-40c4-abb8-5aa42d0f8af9" containerName="extract" Jun 03 16:31:02.654809 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.654785 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:02.657429 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.657406 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 03 16:31:02.659459 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.659437 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674dd879f4-nqz52"] Jun 03 16:31:02.712682 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.712652 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30f11ec5-2887-4a8c-9577-d82d2addc838-tls-cert\") pod \"authorino-674dd879f4-nqz52\" (UID: \"30f11ec5-2887-4a8c-9577-d82d2addc838\") " pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:02.712827 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.712774 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm5dm\" (UniqueName: \"kubernetes.io/projected/30f11ec5-2887-4a8c-9577-d82d2addc838-kube-api-access-vm5dm\") pod \"authorino-674dd879f4-nqz52\" (UID: \"30f11ec5-2887-4a8c-9577-d82d2addc838\") " pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:02.813213 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.813180 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30f11ec5-2887-4a8c-9577-d82d2addc838-tls-cert\") pod \"authorino-674dd879f4-nqz52\" (UID: \"30f11ec5-2887-4a8c-9577-d82d2addc838\") " pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:02.813406 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.813240 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vm5dm\" (UniqueName: \"kubernetes.io/projected/30f11ec5-2887-4a8c-9577-d82d2addc838-kube-api-access-vm5dm\") pod \"authorino-674dd879f4-nqz52\" (UID: \"30f11ec5-2887-4a8c-9577-d82d2addc838\") " pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:02.815584 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.815554 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30f11ec5-2887-4a8c-9577-d82d2addc838-tls-cert\") pod \"authorino-674dd879f4-nqz52\" (UID: \"30f11ec5-2887-4a8c-9577-d82d2addc838\") " pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:02.822603 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.822580 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm5dm\" (UniqueName: \"kubernetes.io/projected/30f11ec5-2887-4a8c-9577-d82d2addc838-kube-api-access-vm5dm\") pod \"authorino-674dd879f4-nqz52\" (UID: \"30f11ec5-2887-4a8c-9577-d82d2addc838\") " pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:02.965535 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:02.965458 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:03.093193 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.093171 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674dd879f4-nqz52"] Jun 03 16:31:03.094657 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:31:03.094633 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30f11ec5_2887_4a8c_9577_d82d2addc838.slice/crio-9dca360305f12651f391e491bcea9856554a521bba7faf2b0bea3166eef63a0b WatchSource:0}: Error finding container 9dca360305f12651f391e491bcea9856554a521bba7faf2b0bea3166eef63a0b: Status 404 returned error can't find the container with id 9dca360305f12651f391e491bcea9856554a521bba7faf2b0bea3166eef63a0b Jun 03 16:31:03.393171 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.393140 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674dd879f4-nqz52"] Jun 03 16:31:03.422780 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.422757 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56fdd757f5-pq86m"] Jun 03 16:31:03.427490 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.427475 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:03.433348 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.433313 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-pq86m"] Jun 03 16:31:03.518649 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.518624 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckl7s\" (UniqueName: \"kubernetes.io/projected/9301b7e8-41b2-46fc-a682-0e90750d677d-kube-api-access-ckl7s\") pod \"authorino-56fdd757f5-pq86m\" (UID: \"9301b7e8-41b2-46fc-a682-0e90750d677d\") " pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:03.518770 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.518752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9301b7e8-41b2-46fc-a682-0e90750d677d-tls-cert\") pod \"authorino-56fdd757f5-pq86m\" (UID: \"9301b7e8-41b2-46fc-a682-0e90750d677d\") " pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:03.579166 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.579146 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674dd879f4-nqz52" event={"ID":"30f11ec5-2887-4a8c-9577-d82d2addc838","Type":"ContainerStarted","Data":"9dca360305f12651f391e491bcea9856554a521bba7faf2b0bea3166eef63a0b"} Jun 03 16:31:03.619242 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.619224 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9301b7e8-41b2-46fc-a682-0e90750d677d-tls-cert\") pod \"authorino-56fdd757f5-pq86m\" (UID: \"9301b7e8-41b2-46fc-a682-0e90750d677d\") " pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:03.619301 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.619257 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ckl7s\" (UniqueName: \"kubernetes.io/projected/9301b7e8-41b2-46fc-a682-0e90750d677d-kube-api-access-ckl7s\") pod \"authorino-56fdd757f5-pq86m\" (UID: \"9301b7e8-41b2-46fc-a682-0e90750d677d\") " pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:03.621283 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.621262 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9301b7e8-41b2-46fc-a682-0e90750d677d-tls-cert\") pod \"authorino-56fdd757f5-pq86m\" (UID: \"9301b7e8-41b2-46fc-a682-0e90750d677d\") " pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:03.627357 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.627336 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckl7s\" (UniqueName: \"kubernetes.io/projected/9301b7e8-41b2-46fc-a682-0e90750d677d-kube-api-access-ckl7s\") pod \"authorino-56fdd757f5-pq86m\" (UID: \"9301b7e8-41b2-46fc-a682-0e90750d677d\") " pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:03.737602 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.737575 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:03.861391 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:03.861356 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-pq86m"] Jun 03 16:31:03.862729 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:31:03.862704 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9301b7e8_41b2_46fc_a682_0e90750d677d.slice/crio-8c21926dd0d977e536cf695f446bae8eae8a30a4483339d7bb4e0a882d6a461a WatchSource:0}: Error finding container 8c21926dd0d977e536cf695f446bae8eae8a30a4483339d7bb4e0a882d6a461a: Status 404 returned error can't find the container with id 8c21926dd0d977e536cf695f446bae8eae8a30a4483339d7bb4e0a882d6a461a Jun 03 16:31:04.583966 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.583930 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674dd879f4-nqz52" event={"ID":"30f11ec5-2887-4a8c-9577-d82d2addc838","Type":"ContainerStarted","Data":"1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f"} Jun 03 16:31:04.584147 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.583993 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-674dd879f4-nqz52" podUID="30f11ec5-2887-4a8c-9577-d82d2addc838" containerName="authorino" containerID="cri-o://1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f" gracePeriod=30 Jun 03 16:31:04.585395 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.585371 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56fdd757f5-pq86m" event={"ID":"9301b7e8-41b2-46fc-a682-0e90750d677d","Type":"ContainerStarted","Data":"f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3"} Jun 03 16:31:04.585501 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.585403 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56fdd757f5-pq86m" event={"ID":"9301b7e8-41b2-46fc-a682-0e90750d677d","Type":"ContainerStarted","Data":"8c21926dd0d977e536cf695f446bae8eae8a30a4483339d7bb4e0a882d6a461a"} Jun 03 16:31:04.609264 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.609226 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-674dd879f4-nqz52" podStartSLOduration=2.09937868 podStartE2EDuration="2.609214751s" podCreationTimestamp="2026-06-03 16:31:02 +0000 UTC" firstStartedPulling="2026-06-03 16:31:03.096259008 +0000 UTC m=+593.360322861" lastFinishedPulling="2026-06-03 16:31:03.606095076 +0000 UTC m=+593.870158932" observedRunningTime="2026-06-03 16:31:04.606450713 +0000 UTC m=+594.870514588" watchObservedRunningTime="2026-06-03 16:31:04.609214751 +0000 UTC m=+594.873278675" Jun 03 16:31:04.657534 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.657490 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-56fdd757f5-pq86m" podStartSLOduration=1.29770112 podStartE2EDuration="1.657478284s" podCreationTimestamp="2026-06-03 16:31:03 +0000 UTC" firstStartedPulling="2026-06-03 16:31:03.864453114 +0000 UTC m=+594.128516967" lastFinishedPulling="2026-06-03 16:31:04.224230278 +0000 UTC m=+594.488294131" observedRunningTime="2026-06-03 16:31:04.627675123 +0000 UTC m=+594.891738999" watchObservedRunningTime="2026-06-03 16:31:04.657478284 +0000 UTC m=+594.921542158" Jun 03 16:31:04.658246 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.658222 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-bfb4n"] Jun 03 16:31:04.658470 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.658436 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-bfb4n" podUID="adbad3a8-1830-443d-8258-fe09018aaeca" containerName="authorino" containerID="cri-o://246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6" gracePeriod=30 Jun 03 16:31:04.922880 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.922848 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:04.926160 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:04.926142 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-bfb4n" Jun 03 16:31:05.030657 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.030631 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm5dm\" (UniqueName: \"kubernetes.io/projected/30f11ec5-2887-4a8c-9577-d82d2addc838-kube-api-access-vm5dm\") pod \"30f11ec5-2887-4a8c-9577-d82d2addc838\" (UID: \"30f11ec5-2887-4a8c-9577-d82d2addc838\") " Jun 03 16:31:05.030847 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.030700 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbt22\" (UniqueName: \"kubernetes.io/projected/adbad3a8-1830-443d-8258-fe09018aaeca-kube-api-access-mbt22\") pod \"adbad3a8-1830-443d-8258-fe09018aaeca\" (UID: \"adbad3a8-1830-443d-8258-fe09018aaeca\") " Jun 03 16:31:05.030847 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.030735 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30f11ec5-2887-4a8c-9577-d82d2addc838-tls-cert\") pod \"30f11ec5-2887-4a8c-9577-d82d2addc838\" (UID: \"30f11ec5-2887-4a8c-9577-d82d2addc838\") " Jun 03 16:31:05.032851 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.032822 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30f11ec5-2887-4a8c-9577-d82d2addc838-kube-api-access-vm5dm" (OuterVolumeSpecName: "kube-api-access-vm5dm") pod "30f11ec5-2887-4a8c-9577-d82d2addc838" (UID: "30f11ec5-2887-4a8c-9577-d82d2addc838"). InnerVolumeSpecName "kube-api-access-vm5dm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:31:05.033045 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.033025 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adbad3a8-1830-443d-8258-fe09018aaeca-kube-api-access-mbt22" (OuterVolumeSpecName: "kube-api-access-mbt22") pod "adbad3a8-1830-443d-8258-fe09018aaeca" (UID: "adbad3a8-1830-443d-8258-fe09018aaeca"). InnerVolumeSpecName "kube-api-access-mbt22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:31:05.040974 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.040953 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30f11ec5-2887-4a8c-9577-d82d2addc838-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "30f11ec5-2887-4a8c-9577-d82d2addc838" (UID: "30f11ec5-2887-4a8c-9577-d82d2addc838"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:31:05.132036 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.131981 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm5dm\" (UniqueName: \"kubernetes.io/projected/30f11ec5-2887-4a8c-9577-d82d2addc838-kube-api-access-vm5dm\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:31:05.132036 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.132003 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbt22\" (UniqueName: \"kubernetes.io/projected/adbad3a8-1830-443d-8258-fe09018aaeca-kube-api-access-mbt22\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:31:05.132036 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.132012 2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30f11ec5-2887-4a8c-9577-d82d2addc838-tls-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:31:05.591172 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.591142 2576 generic.go:358] "Generic (PLEG): container finished" podID="adbad3a8-1830-443d-8258-fe09018aaeca" containerID="246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6" exitCode=0 Jun 03 16:31:05.591361 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.591189 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-bfb4n" Jun 03 16:31:05.591361 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.591227 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-bfb4n" event={"ID":"adbad3a8-1830-443d-8258-fe09018aaeca","Type":"ContainerDied","Data":"246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6"} Jun 03 16:31:05.591361 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.591261 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-bfb4n" event={"ID":"adbad3a8-1830-443d-8258-fe09018aaeca","Type":"ContainerDied","Data":"f79e85146e8c034427381a8f2bf0281ebb8d35f194a1b80fa91f37c8a665ce62"} Jun 03 16:31:05.591361 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.591279 2576 scope.go:117] "RemoveContainer" containerID="246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6" Jun 03 16:31:05.592559 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.592460 2576 generic.go:358] "Generic (PLEG): container finished" podID="30f11ec5-2887-4a8c-9577-d82d2addc838" containerID="1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f" exitCode=0 Jun 03 16:31:05.592559 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.592502 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674dd879f4-nqz52" Jun 03 16:31:05.592559 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.592520 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674dd879f4-nqz52" event={"ID":"30f11ec5-2887-4a8c-9577-d82d2addc838","Type":"ContainerDied","Data":"1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f"} Jun 03 16:31:05.592559 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.592546 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674dd879f4-nqz52" event={"ID":"30f11ec5-2887-4a8c-9577-d82d2addc838","Type":"ContainerDied","Data":"9dca360305f12651f391e491bcea9856554a521bba7faf2b0bea3166eef63a0b"} Jun 03 16:31:05.600522 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.600435 2576 scope.go:117] "RemoveContainer" containerID="246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6" Jun 03 16:31:05.600712 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:31:05.600694 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6\": container with ID starting with 246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6 not found: ID does not exist" containerID="246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6" Jun 03 16:31:05.600759 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.600718 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6"} err="failed to get container status \"246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6\": rpc error: code = NotFound desc = could not find container \"246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6\": container with ID starting with 246a1cd2b2f816399a0df77aab4d3e7170082fafc70cb560c7e6600b0a1e68a6 not found: ID does not exist" Jun 03 16:31:05.600759 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.600735 2576 scope.go:117] "RemoveContainer" containerID="1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f" Jun 03 16:31:05.608947 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.608932 2576 scope.go:117] "RemoveContainer" containerID="1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f" Jun 03 16:31:05.609160 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:31:05.609138 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f\": container with ID starting with 1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f not found: ID does not exist" containerID="1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f" Jun 03 16:31:05.609202 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.609164 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f"} err="failed to get container status \"1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f\": rpc error: code = NotFound desc = could not find container \"1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f\": container with ID starting with 1eb48980f7785506dfa830f78a7bd46feafc7b8ed9abd8d931c4409a33835a1f not found: ID does not exist" Jun 03 16:31:05.615818 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.615798 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-bfb4n"] Jun 03 16:31:05.619790 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.619766 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-bfb4n"] Jun 03 16:31:05.629564 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.629544 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674dd879f4-nqz52"] Jun 03 16:31:05.633771 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.633753 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-674dd879f4-nqz52"] Jun 03 16:31:05.899712 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.899635 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69c4dd8d97-fsxbv"] Jun 03 16:31:05.900001 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.899989 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30f11ec5-2887-4a8c-9577-d82d2addc838" containerName="authorino" Jun 03 16:31:05.900044 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.900001 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="30f11ec5-2887-4a8c-9577-d82d2addc838" containerName="authorino" Jun 03 16:31:05.900044 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.900035 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adbad3a8-1830-443d-8258-fe09018aaeca" containerName="authorino" Jun 03 16:31:05.900044 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.900040 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="adbad3a8-1830-443d-8258-fe09018aaeca" containerName="authorino" Jun 03 16:31:05.900134 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.900101 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="30f11ec5-2887-4a8c-9577-d82d2addc838" containerName="authorino" Jun 03 16:31:05.900134 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.900113 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="adbad3a8-1830-443d-8258-fe09018aaeca" containerName="authorino" Jun 03 16:31:05.904614 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.904596 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:31:05.922538 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:05.922516 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69c4dd8d97-fsxbv"] Jun 03 16:31:06.041761 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.041735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4zhc\" (UniqueName: \"kubernetes.io/projected/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-kube-api-access-s4zhc\") pod \"authorino-69c4dd8d97-fsxbv\" (UID: \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\") " pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:31:06.042088 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.041770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-tls-cert\") pod \"authorino-69c4dd8d97-fsxbv\" (UID: \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\") " pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:31:06.142617 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.142596 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s4zhc\" (UniqueName: \"kubernetes.io/projected/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-kube-api-access-s4zhc\") pod \"authorino-69c4dd8d97-fsxbv\" (UID: \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\") " pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:31:06.142754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.142625 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-tls-cert\") pod \"authorino-69c4dd8d97-fsxbv\" (UID: \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\") " pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:31:06.145001 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.144982 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-tls-cert\") pod \"authorino-69c4dd8d97-fsxbv\" (UID: \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\") " pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:31:06.153999 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.153948 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4zhc\" (UniqueName: \"kubernetes.io/projected/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-kube-api-access-s4zhc\") pod \"authorino-69c4dd8d97-fsxbv\" (UID: \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\") " pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:31:06.213152 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.213131 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:31:06.314632 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.314605 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30f11ec5-2887-4a8c-9577-d82d2addc838" path="/var/lib/kubelet/pods/30f11ec5-2887-4a8c-9577-d82d2addc838/volumes" Jun 03 16:31:06.315078 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.315046 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adbad3a8-1830-443d-8258-fe09018aaeca" path="/var/lib/kubelet/pods/adbad3a8-1830-443d-8258-fe09018aaeca/volumes" Jun 03 16:31:06.345332 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.345290 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69c4dd8d97-fsxbv"] Jun 03 16:31:06.345904 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:31:06.345882 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dcc6e62_2559_4524_a5c5_d0bec416e2ad.slice/crio-7c50210b7327cff9e4b8eec7926e788cb47f27a60506cdd7c21da2dbb873d631 WatchSource:0}: Error finding container 7c50210b7327cff9e4b8eec7926e788cb47f27a60506cdd7c21da2dbb873d631: Status 404 returned error can't find the container with id 7c50210b7327cff9e4b8eec7926e788cb47f27a60506cdd7c21da2dbb873d631 Jun 03 16:31:06.598637 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:06.598603 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" event={"ID":"9dcc6e62-2559-4524-a5c5-d0bec416e2ad","Type":"ContainerStarted","Data":"7c50210b7327cff9e4b8eec7926e788cb47f27a60506cdd7c21da2dbb873d631"} Jun 03 16:31:07.603673 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.603636 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" event={"ID":"9dcc6e62-2559-4524-a5c5-d0bec416e2ad","Type":"ContainerStarted","Data":"d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453"} Jun 03 16:31:07.624413 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.624366 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" podStartSLOduration=2.325357827 podStartE2EDuration="2.624352199s" podCreationTimestamp="2026-06-03 16:31:05 +0000 UTC" firstStartedPulling="2026-06-03 16:31:06.347105216 +0000 UTC m=+596.611169070" lastFinishedPulling="2026-06-03 16:31:06.646099589 +0000 UTC m=+596.910163442" observedRunningTime="2026-06-03 16:31:07.622162823 +0000 UTC m=+597.886226690" watchObservedRunningTime="2026-06-03 16:31:07.624352199 +0000 UTC m=+597.888416074" Jun 03 16:31:07.654089 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.654063 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-pq86m"] Jun 03 16:31:07.654246 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.654229 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-56fdd757f5-pq86m" podUID="9301b7e8-41b2-46fc-a682-0e90750d677d" containerName="authorino" containerID="cri-o://f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3" gracePeriod=30 Jun 03 16:31:07.899937 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.899915 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:07.957949 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.957925 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckl7s\" (UniqueName: \"kubernetes.io/projected/9301b7e8-41b2-46fc-a682-0e90750d677d-kube-api-access-ckl7s\") pod \"9301b7e8-41b2-46fc-a682-0e90750d677d\" (UID: \"9301b7e8-41b2-46fc-a682-0e90750d677d\") " Jun 03 16:31:07.958050 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.957974 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9301b7e8-41b2-46fc-a682-0e90750d677d-tls-cert\") pod \"9301b7e8-41b2-46fc-a682-0e90750d677d\" (UID: \"9301b7e8-41b2-46fc-a682-0e90750d677d\") " Jun 03 16:31:07.959853 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.959832 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9301b7e8-41b2-46fc-a682-0e90750d677d-kube-api-access-ckl7s" (OuterVolumeSpecName: "kube-api-access-ckl7s") pod "9301b7e8-41b2-46fc-a682-0e90750d677d" (UID: "9301b7e8-41b2-46fc-a682-0e90750d677d"). InnerVolumeSpecName "kube-api-access-ckl7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:31:07.968077 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:07.968047 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9301b7e8-41b2-46fc-a682-0e90750d677d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9301b7e8-41b2-46fc-a682-0e90750d677d" (UID: "9301b7e8-41b2-46fc-a682-0e90750d677d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:31:08.058986 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.058956 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckl7s\" (UniqueName: \"kubernetes.io/projected/9301b7e8-41b2-46fc-a682-0e90750d677d-kube-api-access-ckl7s\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:31:08.058986 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.058985 2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9301b7e8-41b2-46fc-a682-0e90750d677d-tls-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:31:08.608640 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.608543 2576 generic.go:358] "Generic (PLEG): container finished" podID="9301b7e8-41b2-46fc-a682-0e90750d677d" containerID="f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3" exitCode=0 Jun 03 16:31:08.608640 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.608597 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-pq86m" Jun 03 16:31:08.608640 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.608628 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56fdd757f5-pq86m" event={"ID":"9301b7e8-41b2-46fc-a682-0e90750d677d","Type":"ContainerDied","Data":"f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3"} Jun 03 16:31:08.609163 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.608664 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56fdd757f5-pq86m" event={"ID":"9301b7e8-41b2-46fc-a682-0e90750d677d","Type":"ContainerDied","Data":"8c21926dd0d977e536cf695f446bae8eae8a30a4483339d7bb4e0a882d6a461a"} Jun 03 16:31:08.609163 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.608680 2576 scope.go:117] "RemoveContainer" containerID="f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3" Jun 03 16:31:08.617270 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.617251 2576 scope.go:117] "RemoveContainer" containerID="f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3" Jun 03 16:31:08.617524 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:31:08.617503 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3\": container with ID starting with f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3 not found: ID does not exist" containerID="f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3" Jun 03 16:31:08.617597 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.617531 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3"} err="failed to get container status \"f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3\": rpc error: code = NotFound desc = could not find container \"f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3\": container with ID starting with f0897dfe64601d67fde694237a9aceffcf7c545ffcbc42c67149e65bbb66fea3 not found: ID does not exist" Jun 03 16:31:08.626828 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.626807 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-pq86m"] Jun 03 16:31:08.631180 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:08.631159 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-pq86m"] Jun 03 16:31:10.319593 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:10.319555 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9301b7e8-41b2-46fc-a682-0e90750d677d" path="/var/lib/kubelet/pods/9301b7e8-41b2-46fc-a682-0e90750d677d/volumes" Jun 03 16:31:55.697969 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.697936 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-746cc5ddcf-np887"] Jun 03 16:31:55.698414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.698332 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9301b7e8-41b2-46fc-a682-0e90750d677d" containerName="authorino" Jun 03 16:31:55.698414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.698343 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9301b7e8-41b2-46fc-a682-0e90750d677d" containerName="authorino" Jun 03 16:31:55.698414 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.698414 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="9301b7e8-41b2-46fc-a682-0e90750d677d" containerName="authorino" Jun 03 16:31:55.701284 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.701270 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:55.704085 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.704064 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 03 16:31:55.704394 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.704370 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-s9nms\"" Jun 03 16:31:55.705140 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.705126 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 03 16:31:55.713202 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.713180 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746cc5ddcf-np887"] Jun 03 16:31:55.765703 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.765672 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzz9g\" (UniqueName: \"kubernetes.io/projected/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-kube-api-access-zzz9g\") pod \"maas-controller-746cc5ddcf-np887\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:55.765703 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.765717 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert\") pod \"maas-controller-746cc5ddcf-np887\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:55.866620 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.866578 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zzz9g\" (UniqueName: \"kubernetes.io/projected/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-kube-api-access-zzz9g\") pod \"maas-controller-746cc5ddcf-np887\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:55.866788 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.866631 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert\") pod \"maas-controller-746cc5ddcf-np887\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:55.866788 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:31:55.866763 2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 03 16:31:55.866863 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:31:55.866815 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert podName:6f5efe95-0dae-4cb4-9eff-86cf75d58bc2 nodeName:}" failed. No retries permitted until 2026-06-03 16:31:56.3667986 +0000 UTC m=+646.630862453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert") pod "maas-controller-746cc5ddcf-np887" (UID: "6f5efe95-0dae-4cb4-9eff-86cf75d58bc2") : secret "maas-controller-webhook-cert" not found Jun 03 16:31:55.876542 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:55.876512 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzz9g\" (UniqueName: \"kubernetes.io/projected/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-kube-api-access-zzz9g\") pod \"maas-controller-746cc5ddcf-np887\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:56.372649 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.372609 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert\") pod \"maas-controller-746cc5ddcf-np887\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:56.374950 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.374921 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert\") pod \"maas-controller-746cc5ddcf-np887\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:56.612172 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.612139 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:56.749233 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.749209 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746cc5ddcf-np887"] Jun 03 16:31:56.750388 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:31:56.750364 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5efe95_0dae_4cb4_9eff_86cf75d58bc2.slice/crio-3da17f5b210a1854df462f8dbc8fd97f694b5abf53f379b0eb32e57898c16a93 WatchSource:0}: Error finding container 3da17f5b210a1854df462f8dbc8fd97f694b5abf53f379b0eb32e57898c16a93: Status 404 returned error can't find the container with id 3da17f5b210a1854df462f8dbc8fd97f694b5abf53f379b0eb32e57898c16a93 Jun 03 16:31:56.751661 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.751642 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:31:56.802634 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.802604 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746cc5ddcf-np887" event={"ID":"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2","Type":"ContainerStarted","Data":"3da17f5b210a1854df462f8dbc8fd97f694b5abf53f379b0eb32e57898c16a93"} Jun 03 16:31:56.946999 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.946915 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f579569d9-nrpcv"] Jun 03 16:31:56.951997 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.951976 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:56.959296 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:56.959271 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f579569d9-nrpcv"] Jun 03 16:31:57.079746 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.079704 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkqwg\" (UniqueName: \"kubernetes.io/projected/399e7a68-b887-4d39-8cc9-9fdf79bcbc88-kube-api-access-rkqwg\") pod \"maas-controller-f579569d9-nrpcv\" (UID: \"399e7a68-b887-4d39-8cc9-9fdf79bcbc88\") " pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:57.079920 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.079794 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/399e7a68-b887-4d39-8cc9-9fdf79bcbc88-webhook-cert\") pod \"maas-controller-f579569d9-nrpcv\" (UID: \"399e7a68-b887-4d39-8cc9-9fdf79bcbc88\") " pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:57.180220 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.180180 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rkqwg\" (UniqueName: \"kubernetes.io/projected/399e7a68-b887-4d39-8cc9-9fdf79bcbc88-kube-api-access-rkqwg\") pod \"maas-controller-f579569d9-nrpcv\" (UID: \"399e7a68-b887-4d39-8cc9-9fdf79bcbc88\") " pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:57.180409 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.180250 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/399e7a68-b887-4d39-8cc9-9fdf79bcbc88-webhook-cert\") pod \"maas-controller-f579569d9-nrpcv\" (UID: \"399e7a68-b887-4d39-8cc9-9fdf79bcbc88\") " pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:57.182595 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.182577 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/399e7a68-b887-4d39-8cc9-9fdf79bcbc88-webhook-cert\") pod \"maas-controller-f579569d9-nrpcv\" (UID: \"399e7a68-b887-4d39-8cc9-9fdf79bcbc88\") " pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:57.189666 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.189640 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkqwg\" (UniqueName: \"kubernetes.io/projected/399e7a68-b887-4d39-8cc9-9fdf79bcbc88-kube-api-access-rkqwg\") pod \"maas-controller-f579569d9-nrpcv\" (UID: \"399e7a68-b887-4d39-8cc9-9fdf79bcbc88\") " pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:57.263412 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.263373 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:57.418874 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.418846 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f579569d9-nrpcv"] Jun 03 16:31:57.421051 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:31:57.421020 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399e7a68_b887_4d39_8cc9_9fdf79bcbc88.slice/crio-65c792dd100fd826da8e8a820009e23bca1a9a37b9f1a45e229fb5662e6ddf1a WatchSource:0}: Error finding container 65c792dd100fd826da8e8a820009e23bca1a9a37b9f1a45e229fb5662e6ddf1a: Status 404 returned error can't find the container with id 65c792dd100fd826da8e8a820009e23bca1a9a37b9f1a45e229fb5662e6ddf1a Jun 03 16:31:57.810867 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:57.810831 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f579569d9-nrpcv" event={"ID":"399e7a68-b887-4d39-8cc9-9fdf79bcbc88","Type":"ContainerStarted","Data":"65c792dd100fd826da8e8a820009e23bca1a9a37b9f1a45e229fb5662e6ddf1a"} Jun 03 16:31:59.820793 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:59.820752 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f579569d9-nrpcv" event={"ID":"399e7a68-b887-4d39-8cc9-9fdf79bcbc88","Type":"ContainerStarted","Data":"2ff86cddc272bdd55c9ba8cb556e87ebc0e085cebf174ee3b1f9908067e5c5d0"} Jun 03 16:31:59.821263 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:59.820828 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:31:59.822169 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:59.822148 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746cc5ddcf-np887" event={"ID":"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2","Type":"ContainerStarted","Data":"15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196"} Jun 03 16:31:59.822352 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:59.822300 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:31:59.843080 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:59.843030 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-f579569d9-nrpcv" podStartSLOduration=1.910851483 podStartE2EDuration="3.843015695s" podCreationTimestamp="2026-06-03 16:31:56 +0000 UTC" firstStartedPulling="2026-06-03 16:31:57.422719444 +0000 UTC m=+647.686783304" lastFinishedPulling="2026-06-03 16:31:59.354883662 +0000 UTC m=+649.618947516" observedRunningTime="2026-06-03 16:31:59.840774418 +0000 UTC m=+650.104838295" watchObservedRunningTime="2026-06-03 16:31:59.843015695 +0000 UTC m=+650.107079569" Jun 03 16:31:59.862873 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:31:59.862819 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-746cc5ddcf-np887" podStartSLOduration=2.267040202 podStartE2EDuration="4.862805666s" podCreationTimestamp="2026-06-03 16:31:55 +0000 UTC" firstStartedPulling="2026-06-03 16:31:56.752194456 +0000 UTC m=+647.016258324" lastFinishedPulling="2026-06-03 16:31:59.347959929 +0000 UTC m=+649.612023788" observedRunningTime="2026-06-03 16:31:59.860478992 +0000 UTC m=+650.124542869" watchObservedRunningTime="2026-06-03 16:31:59.862805666 +0000 UTC m=+650.126869540" Jun 03 16:32:10.833820 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:10.833791 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f579569d9-nrpcv" Jun 03 16:32:10.834261 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:10.833848 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:32:10.906272 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:10.906235 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-746cc5ddcf-np887"] Jun 03 16:32:10.906538 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:10.906511 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-746cc5ddcf-np887" podUID="6f5efe95-0dae-4cb4-9eff-86cf75d58bc2" containerName="manager" containerID="cri-o://15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196" gracePeriod=30 Jun 03 16:32:11.150457 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.150434 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:32:11.306518 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.306485 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert\") pod \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " Jun 03 16:32:11.306684 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.306575 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzz9g\" (UniqueName: \"kubernetes.io/projected/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-kube-api-access-zzz9g\") pod \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\" (UID: \"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2\") " Jun 03 16:32:11.308696 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.308672 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6f5efe95-0dae-4cb4-9eff-86cf75d58bc2" (UID: "6f5efe95-0dae-4cb4-9eff-86cf75d58bc2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:32:11.308807 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.308782 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-kube-api-access-zzz9g" (OuterVolumeSpecName: "kube-api-access-zzz9g") pod "6f5efe95-0dae-4cb4-9eff-86cf75d58bc2" (UID: "6f5efe95-0dae-4cb4-9eff-86cf75d58bc2"). InnerVolumeSpecName "kube-api-access-zzz9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:32:11.407440 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.407358 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-webhook-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:32:11.407440 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.407388 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzz9g\" (UniqueName: \"kubernetes.io/projected/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2-kube-api-access-zzz9g\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:32:11.872245 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.872214 2576 generic.go:358] "Generic (PLEG): container finished" podID="6f5efe95-0dae-4cb4-9eff-86cf75d58bc2" containerID="15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196" exitCode=0 Jun 03 16:32:11.872754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.872254 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746cc5ddcf-np887" event={"ID":"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2","Type":"ContainerDied","Data":"15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196"} Jun 03 16:32:11.872754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.872285 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746cc5ddcf-np887" Jun 03 16:32:11.872754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.872292 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746cc5ddcf-np887" event={"ID":"6f5efe95-0dae-4cb4-9eff-86cf75d58bc2","Type":"ContainerDied","Data":"3da17f5b210a1854df462f8dbc8fd97f694b5abf53f379b0eb32e57898c16a93"} Jun 03 16:32:11.872754 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.872308 2576 scope.go:117] "RemoveContainer" containerID="15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196" Jun 03 16:32:11.883507 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.882875 2576 scope.go:117] "RemoveContainer" containerID="15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196" Jun 03 16:32:11.884069 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:32:11.884050 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196\": container with ID starting with 15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196 not found: ID does not exist" containerID="15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196" Jun 03 16:32:11.884145 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.884077 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196"} err="failed to get container status \"15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196\": rpc error: code = NotFound desc = could not find container \"15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196\": container with ID starting with 15402564a5fc786ffe2a07ce8abf099509b112916e01aa6ba10f1283d78f7196 not found: ID does not exist" Jun 03 16:32:11.895734 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.895710 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-746cc5ddcf-np887"] Jun 03 16:32:11.899910 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:11.899887 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-746cc5ddcf-np887"] Jun 03 16:32:12.314338 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:12.314293 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f5efe95-0dae-4cb4-9eff-86cf75d58bc2" path="/var/lib/kubelet/pods/6f5efe95-0dae-4cb4-9eff-86cf75d58bc2/volumes" Jun 03 16:32:28.001306 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.001268 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6c84f6ccdc-7gp92"] Jun 03 16:32:28.001942 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.001922 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5efe95-0dae-4cb4-9eff-86cf75d58bc2" containerName="manager" Jun 03 16:32:28.002021 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.001946 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5efe95-0dae-4cb4-9eff-86cf75d58bc2" containerName="manager" Jun 03 16:32:28.002073 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.002047 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="6f5efe95-0dae-4cb4-9eff-86cf75d58bc2" containerName="manager" Jun 03 16:32:28.005317 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.005297 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.009892 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.009871 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 03 16:32:28.010031 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.009908 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-prkdl\"" Jun 03 16:32:28.020353 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.020292 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6c84f6ccdc-7gp92"] Jun 03 16:32:28.049287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.049247 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-maas-api-tls\") pod \"maas-api-6c84f6ccdc-7gp92\" (UID: \"a55ef0f5-8a9d-410a-90f1-d55cffec34a2\") " pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.049287 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.049285 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-942pc\" (UniqueName: \"kubernetes.io/projected/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-kube-api-access-942pc\") pod \"maas-api-6c84f6ccdc-7gp92\" (UID: \"a55ef0f5-8a9d-410a-90f1-d55cffec34a2\") " pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.150057 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.150022 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-maas-api-tls\") pod \"maas-api-6c84f6ccdc-7gp92\" (UID: \"a55ef0f5-8a9d-410a-90f1-d55cffec34a2\") " pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.150057 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.150055 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-942pc\" (UniqueName: \"kubernetes.io/projected/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-kube-api-access-942pc\") pod \"maas-api-6c84f6ccdc-7gp92\" (UID: \"a55ef0f5-8a9d-410a-90f1-d55cffec34a2\") " pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.150264 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:32:28.150166 2576 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 03 16:32:28.150264 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:32:28.150244 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-maas-api-tls podName:a55ef0f5-8a9d-410a-90f1-d55cffec34a2 nodeName:}" failed. No retries permitted until 2026-06-03 16:32:28.650221725 +0000 UTC m=+678.914285582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-maas-api-tls") pod "maas-api-6c84f6ccdc-7gp92" (UID: "a55ef0f5-8a9d-410a-90f1-d55cffec34a2") : secret "maas-api-serving-cert" not found Jun 03 16:32:28.159769 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.159739 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-942pc\" (UniqueName: \"kubernetes.io/projected/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-kube-api-access-942pc\") pod \"maas-api-6c84f6ccdc-7gp92\" (UID: \"a55ef0f5-8a9d-410a-90f1-d55cffec34a2\") " pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.413602 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.413565 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-dhnd2"] Jun 03 16:32:28.417844 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.417818 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" Jun 03 16:32:28.420677 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.420651 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wqkxv\"" Jun 03 16:32:28.420677 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.420672 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 03 16:32:28.430580 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.430550 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-dhnd2"] Jun 03 16:32:28.452724 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.452684 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ltxn\" (UniqueName: \"kubernetes.io/projected/49cc4104-4b08-49d7-a025-ef3c6c440a28-kube-api-access-6ltxn\") pod \"payload-processing-58b6c8fdc7-dhnd2\" (UID: \"49cc4104-4b08-49d7-a025-ef3c6c440a28\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" Jun 03 16:32:28.554129 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.554088 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6ltxn\" (UniqueName: \"kubernetes.io/projected/49cc4104-4b08-49d7-a025-ef3c6c440a28-kube-api-access-6ltxn\") pod \"payload-processing-58b6c8fdc7-dhnd2\" (UID: \"49cc4104-4b08-49d7-a025-ef3c6c440a28\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" Jun 03 16:32:28.569452 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.569427 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ltxn\" (UniqueName: \"kubernetes.io/projected/49cc4104-4b08-49d7-a025-ef3c6c440a28-kube-api-access-6ltxn\") pod \"payload-processing-58b6c8fdc7-dhnd2\" (UID: \"49cc4104-4b08-49d7-a025-ef3c6c440a28\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" Jun 03 16:32:28.654603 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.654569 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-maas-api-tls\") pod \"maas-api-6c84f6ccdc-7gp92\" (UID: \"a55ef0f5-8a9d-410a-90f1-d55cffec34a2\") " pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.657057 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.657029 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a55ef0f5-8a9d-410a-90f1-d55cffec34a2-maas-api-tls\") pod \"maas-api-6c84f6ccdc-7gp92\" (UID: \"a55ef0f5-8a9d-410a-90f1-d55cffec34a2\") " pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.730761 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.730677 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" Jun 03 16:32:28.876986 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.876959 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-dhnd2"] Jun 03 16:32:28.879269 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:32:28.879242 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49cc4104_4b08_49d7_a025_ef3c6c440a28.slice/crio-b15d27a7a9b525dc633b5185d96911ac18ff9dacd84cbdba63a26b5da3eb939a WatchSource:0}: Error finding container b15d27a7a9b525dc633b5185d96911ac18ff9dacd84cbdba63a26b5da3eb939a: Status 404 returned error can't find the container with id b15d27a7a9b525dc633b5185d96911ac18ff9dacd84cbdba63a26b5da3eb939a Jun 03 16:32:28.916475 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.916443 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:28.938014 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:28.937866 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" event={"ID":"49cc4104-4b08-49d7-a025-ef3c6c440a28","Type":"ContainerStarted","Data":"b15d27a7a9b525dc633b5185d96911ac18ff9dacd84cbdba63a26b5da3eb939a"} Jun 03 16:32:29.059735 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:29.059707 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6c84f6ccdc-7gp92"] Jun 03 16:32:29.065477 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:32:29.063671 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda55ef0f5_8a9d_410a_90f1_d55cffec34a2.slice/crio-0d766792f273b612a3e82844f493eb2558ceee4ec11145a543c0a5806c1c1603 WatchSource:0}: Error finding container 0d766792f273b612a3e82844f493eb2558ceee4ec11145a543c0a5806c1c1603: Status 404 returned error can't find the container with id 0d766792f273b612a3e82844f493eb2558ceee4ec11145a543c0a5806c1c1603 Jun 03 16:32:29.946808 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:29.946764 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6c84f6ccdc-7gp92" event={"ID":"a55ef0f5-8a9d-410a-90f1-d55cffec34a2","Type":"ContainerStarted","Data":"0d766792f273b612a3e82844f493eb2558ceee4ec11145a543c0a5806c1c1603"} Jun 03 16:32:31.956664 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:31.956627 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" event={"ID":"49cc4104-4b08-49d7-a025-ef3c6c440a28","Type":"ContainerStarted","Data":"300bea7252ebaa709f806394922ea11c7184bd6a665db29ee3309e6e77a8b0cb"} Jun 03 16:32:31.957134 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:31.956683 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" Jun 03 16:32:31.958100 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:31.958071 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6c84f6ccdc-7gp92" event={"ID":"a55ef0f5-8a9d-410a-90f1-d55cffec34a2","Type":"ContainerStarted","Data":"cf67bc2341dcfca16a71ad55a8469b96487299808a03db00b1c97db5d1f76ae7"} Jun 03 16:32:31.958212 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:31.958193 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:31.980061 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:31.980009 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" podStartSLOduration=1.4121146709999999 podStartE2EDuration="3.97999384s" podCreationTimestamp="2026-06-03 16:32:28 +0000 UTC" firstStartedPulling="2026-06-03 16:32:28.881122678 +0000 UTC m=+679.145186531" lastFinishedPulling="2026-06-03 16:32:31.449001846 +0000 UTC m=+681.713065700" observedRunningTime="2026-06-03 16:32:31.977003338 +0000 UTC m=+682.241067236" watchObservedRunningTime="2026-06-03 16:32:31.97999384 +0000 UTC m=+682.244057715" Jun 03 16:32:31.998846 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:31.998794 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-6c84f6ccdc-7gp92" podStartSLOduration=2.615601008 podStartE2EDuration="4.998778957s" podCreationTimestamp="2026-06-03 16:32:27 +0000 UTC" firstStartedPulling="2026-06-03 16:32:29.06584074 +0000 UTC m=+679.329904594" lastFinishedPulling="2026-06-03 16:32:31.44901869 +0000 UTC m=+681.713082543" observedRunningTime="2026-06-03 16:32:31.997905686 +0000 UTC m=+682.261969563" watchObservedRunningTime="2026-06-03 16:32:31.998778957 +0000 UTC m=+682.262842831" Jun 03 16:32:37.970186 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:37.970159 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6c84f6ccdc-7gp92" Jun 03 16:32:42.965107 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:32:42.965072 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-dhnd2" Jun 03 16:33:17.736594 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.736558 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bf6964f95-vcsbx"] Jun 03 16:33:17.744722 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.744695 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.748662 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.748633 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 03 16:33:17.753303 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.753272 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bf6964f95-vcsbx"] Jun 03 16:33:17.856778 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.856734 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0aaec692-9fa6-430b-8b8a-9c26f47172b9-tls-cert\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.856956 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.856788 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wkjj\" (UniqueName: \"kubernetes.io/projected/0aaec692-9fa6-430b-8b8a-9c26f47172b9-kube-api-access-6wkjj\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.856956 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.856879 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0aaec692-9fa6-430b-8b8a-9c26f47172b9-oidc-ca\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.957788 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.957753 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0aaec692-9fa6-430b-8b8a-9c26f47172b9-tls-cert\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.957953 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.957798 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6wkjj\" (UniqueName: \"kubernetes.io/projected/0aaec692-9fa6-430b-8b8a-9c26f47172b9-kube-api-access-6wkjj\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.957953 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.957828 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0aaec692-9fa6-430b-8b8a-9c26f47172b9-oidc-ca\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.958499 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.958480 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0aaec692-9fa6-430b-8b8a-9c26f47172b9-oidc-ca\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.960299 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.960280 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0aaec692-9fa6-430b-8b8a-9c26f47172b9-tls-cert\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:17.966710 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:17.966688 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wkjj\" (UniqueName: \"kubernetes.io/projected/0aaec692-9fa6-430b-8b8a-9c26f47172b9-kube-api-access-6wkjj\") pod \"authorino-5bf6964f95-vcsbx\" (UID: \"0aaec692-9fa6-430b-8b8a-9c26f47172b9\") " pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:18.054980 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:18.054954 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bf6964f95-vcsbx" Jun 03 16:33:18.187881 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:18.187854 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bf6964f95-vcsbx"] Jun 03 16:33:18.188652 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:33:18.188615 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aaec692_9fa6_430b_8b8a_9c26f47172b9.slice/crio-8da19b73e3e028d910024c6f253dd5337d4d14fbca017050a79d967381eb6da6 WatchSource:0}: Error finding container 8da19b73e3e028d910024c6f253dd5337d4d14fbca017050a79d967381eb6da6: Status 404 returned error can't find the container with id 8da19b73e3e028d910024c6f253dd5337d4d14fbca017050a79d967381eb6da6 Jun 03 16:33:19.159743 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.159710 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bf6964f95-vcsbx" event={"ID":"0aaec692-9fa6-430b-8b8a-9c26f47172b9","Type":"ContainerStarted","Data":"2424879079ba12423c6b4bb8c5ba0c0c9b95beff9e16fb99c082fed7328a020b"} Jun 03 16:33:19.160105 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.159750 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bf6964f95-vcsbx" event={"ID":"0aaec692-9fa6-430b-8b8a-9c26f47172b9","Type":"ContainerStarted","Data":"8da19b73e3e028d910024c6f253dd5337d4d14fbca017050a79d967381eb6da6"} Jun 03 16:33:19.188839 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.188789 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-5bf6964f95-vcsbx" podStartSLOduration=1.749655031 podStartE2EDuration="2.188772131s" podCreationTimestamp="2026-06-03 16:33:17 +0000 UTC" firstStartedPulling="2026-06-03 16:33:18.189872051 +0000 UTC m=+728.453935905" lastFinishedPulling="2026-06-03 16:33:18.628989152 +0000 UTC m=+728.893053005" observedRunningTime="2026-06-03 16:33:19.186465095 +0000 UTC m=+729.450528972" watchObservedRunningTime="2026-06-03 16:33:19.188772131 +0000 UTC m=+729.452836037" Jun 03 16:33:19.231659 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.231629 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69c4dd8d97-fsxbv"] Jun 03 16:33:19.231840 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.231819 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" podUID="9dcc6e62-2559-4524-a5c5-d0bec416e2ad" containerName="authorino" containerID="cri-o://d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453" gracePeriod=30 Jun 03 16:33:19.492007 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.491978 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:33:19.572057 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.572023 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-tls-cert\") pod \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\" (UID: \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\") " Jun 03 16:33:19.572215 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.572110 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4zhc\" (UniqueName: \"kubernetes.io/projected/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-kube-api-access-s4zhc\") pod \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\" (UID: \"9dcc6e62-2559-4524-a5c5-d0bec416e2ad\") " Jun 03 16:33:19.574169 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.574136 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-kube-api-access-s4zhc" (OuterVolumeSpecName: "kube-api-access-s4zhc") pod "9dcc6e62-2559-4524-a5c5-d0bec416e2ad" (UID: "9dcc6e62-2559-4524-a5c5-d0bec416e2ad"). InnerVolumeSpecName "kube-api-access-s4zhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:33:19.583021 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.582997 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9dcc6e62-2559-4524-a5c5-d0bec416e2ad" (UID: "9dcc6e62-2559-4524-a5c5-d0bec416e2ad"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:33:19.673911 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.673874 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4zhc\" (UniqueName: \"kubernetes.io/projected/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-kube-api-access-s4zhc\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:33:19.673911 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:19.673914 2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9dcc6e62-2559-4524-a5c5-d0bec416e2ad-tls-cert\") on node \"ip-10-0-128-141.ec2.internal\" DevicePath \"\"" Jun 03 16:33:20.166102 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.166068 2576 generic.go:358] "Generic (PLEG): container finished" podID="9dcc6e62-2559-4524-a5c5-d0bec416e2ad" containerID="d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453" exitCode=0 Jun 03 16:33:20.166592 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.166126 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" Jun 03 16:33:20.166592 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.166157 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" event={"ID":"9dcc6e62-2559-4524-a5c5-d0bec416e2ad","Type":"ContainerDied","Data":"d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453"} Jun 03 16:33:20.166592 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.166203 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c4dd8d97-fsxbv" event={"ID":"9dcc6e62-2559-4524-a5c5-d0bec416e2ad","Type":"ContainerDied","Data":"7c50210b7327cff9e4b8eec7926e788cb47f27a60506cdd7c21da2dbb873d631"} Jun 03 16:33:20.166592 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.166224 2576 scope.go:117] "RemoveContainer" containerID="d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453" Jun 03 16:33:20.175444 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.175426 2576 scope.go:117] "RemoveContainer" containerID="d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453" Jun 03 16:33:20.175716 ip-10-0-128-141 kubenswrapper[2576]: E0603 16:33:20.175698 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453\": container with ID starting with d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453 not found: ID does not exist" containerID="d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453" Jun 03 16:33:20.175767 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.175724 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453"} err="failed to get container status \"d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453\": rpc error: code = NotFound desc = could not find container \"d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453\": container with ID starting with d0ed88085290ddf6d17340b66923d9b301496991ca8a9f2852ebf626178ec453 not found: ID does not exist" Jun 03 16:33:20.210690 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.210661 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69c4dd8d97-fsxbv"] Jun 03 16:33:20.213256 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.213234 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69c4dd8d97-fsxbv"] Jun 03 16:33:20.315648 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:20.315619 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dcc6e62-2559-4524-a5c5-d0bec416e2ad" path="/var/lib/kubelet/pods/9dcc6e62-2559-4524-a5c5-d0bec416e2ad/volumes" Jun 03 16:33:44.204835 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.204797 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27"] Jun 03 16:33:44.205488 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.205468 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dcc6e62-2559-4524-a5c5-d0bec416e2ad" containerName="authorino" Jun 03 16:33:44.205545 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.205492 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dcc6e62-2559-4524-a5c5-d0bec416e2ad" containerName="authorino" Jun 03 16:33:44.205601 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.205591 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="9dcc6e62-2559-4524-a5c5-d0bec416e2ad" containerName="authorino" Jun 03 16:33:44.210806 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.210784 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.213512 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.213493 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 03 16:33:44.214504 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.214488 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 03 16:33:44.214592 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.214510 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jx46g\"" Jun 03 16:33:44.214592 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.214495 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 03 16:33:44.223505 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.223475 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27"] Jun 03 16:33:44.273693 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.273655 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.273963 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.273943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.274033 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.273972 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.274033 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.274022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.274149 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.274056 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwzp6\" (UniqueName: \"kubernetes.io/projected/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-kube-api-access-dwzp6\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.274149 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.274102 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.374918 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.374881 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.375097 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.374926 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dwzp6\" (UniqueName: \"kubernetes.io/projected/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-kube-api-access-dwzp6\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.375097 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.374953 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.375097 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.374977 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.375097 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.375069 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.375345 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.375118 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.375472 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.375453 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.375629 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.375608 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.375766 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.375750 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.377522 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.377491 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.377670 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.377642 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.384077 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.384055 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwzp6\" (UniqueName: \"kubernetes.io/projected/406c3b4d-d7e0-4f14-8865-3afc17f5ff99-kube-api-access-dwzp6\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27\" (UID: \"406c3b4d-d7e0-4f14-8865-3afc17f5ff99\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.524285 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.524252 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:33:44.669826 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:44.669799 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27"] Jun 03 16:33:44.671419 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:33:44.671391 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod406c3b4d_d7e0_4f14_8865_3afc17f5ff99.slice/crio-09592e315bfea78d2a2903a0d767b0ca7e49590785bb6ed69a0922595079537d WatchSource:0}: Error finding container 09592e315bfea78d2a2903a0d767b0ca7e49590785bb6ed69a0922595079537d: Status 404 returned error can't find the container with id 09592e315bfea78d2a2903a0d767b0ca7e49590785bb6ed69a0922595079537d Jun 03 16:33:45.280121 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:45.280086 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" event={"ID":"406c3b4d-d7e0-4f14-8865-3afc17f5ff99","Type":"ContainerStarted","Data":"09592e315bfea78d2a2903a0d767b0ca7e49590785bb6ed69a0922595079537d"} Jun 03 16:33:50.301962 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:50.301928 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" event={"ID":"406c3b4d-d7e0-4f14-8865-3afc17f5ff99","Type":"ContainerStarted","Data":"b7f74479ce9c3217db721869ddd2adb7bd70e245877efe2593e92428ed1fd144"} Jun 03 16:33:58.332832 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:58.332799 2576 generic.go:358] "Generic (PLEG): container finished" podID="406c3b4d-d7e0-4f14-8865-3afc17f5ff99" containerID="b7f74479ce9c3217db721869ddd2adb7bd70e245877efe2593e92428ed1fd144" exitCode=0 Jun 03 16:33:58.333232 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:33:58.332871 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" event={"ID":"406c3b4d-d7e0-4f14-8865-3afc17f5ff99","Type":"ContainerDied","Data":"b7f74479ce9c3217db721869ddd2adb7bd70e245877efe2593e92428ed1fd144"} Jun 03 16:34:00.343373 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:00.343336 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" event={"ID":"406c3b4d-d7e0-4f14-8865-3afc17f5ff99","Type":"ContainerStarted","Data":"53afe95b7c9fd501212e11ead4f7bf1b2c2662ff350c6f7fb9d964baf6903ef6"} Jun 03 16:34:00.366798 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:00.366750 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" podStartSLOduration=1.438608477 podStartE2EDuration="16.366735621s" podCreationTimestamp="2026-06-03 16:33:44 +0000 UTC" firstStartedPulling="2026-06-03 16:33:44.67306875 +0000 UTC m=+754.937132604" lastFinishedPulling="2026-06-03 16:33:59.601195895 +0000 UTC m=+769.865259748" observedRunningTime="2026-06-03 16:34:00.36295727 +0000 UTC m=+770.627021145" watchObservedRunningTime="2026-06-03 16:34:00.366735621 +0000 UTC m=+770.630799496" Jun 03 16:34:02.795376 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.795344 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv"] Jun 03 16:34:02.813314 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.813286 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv"] Jun 03 16:34:02.813535 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.813466 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:02.816520 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.816496 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 03 16:34:02.933519 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.933490 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b60d34f4-46cf-4530-91e1-7626ca6b4169-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:02.933678 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.933538 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:02.933678 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.933565 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm5g4\" (UniqueName: \"kubernetes.io/projected/b60d34f4-46cf-4530-91e1-7626ca6b4169-kube-api-access-mm5g4\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:02.933678 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.933652 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:02.933849 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.933719 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:02.933849 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:02.933821 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.034886 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.034854 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.035062 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.034902 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b60d34f4-46cf-4530-91e1-7626ca6b4169-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.035062 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.034978 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.035062 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.035024 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mm5g4\" (UniqueName: \"kubernetes.io/projected/b60d34f4-46cf-4530-91e1-7626ca6b4169-kube-api-access-mm5g4\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.035261 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.035068 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.035261 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.035127 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.035387 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.035281 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.035443 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.035407 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.035587 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.035564 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.037396 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.037373 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b60d34f4-46cf-4530-91e1-7626ca6b4169-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.037876 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.037856 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b60d34f4-46cf-4530-91e1-7626ca6b4169-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.043865 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.043833 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm5g4\" (UniqueName: \"kubernetes.io/projected/b60d34f4-46cf-4530-91e1-7626ca6b4169-kube-api-access-mm5g4\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv\" (UID: \"b60d34f4-46cf-4530-91e1-7626ca6b4169\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.128357 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.128289 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:03.268532 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.268503 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv"] Jun 03 16:34:03.269493 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:34:03.269468 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60d34f4_46cf_4530_91e1_7626ca6b4169.slice/crio-58be976574f0af803fcf268bc2d9a6781753c246fc46870e1cbf6e560d6259fb WatchSource:0}: Error finding container 58be976574f0af803fcf268bc2d9a6781753c246fc46870e1cbf6e560d6259fb: Status 404 returned error can't find the container with id 58be976574f0af803fcf268bc2d9a6781753c246fc46870e1cbf6e560d6259fb Jun 03 16:34:03.356237 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.356207 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" event={"ID":"b60d34f4-46cf-4530-91e1-7626ca6b4169","Type":"ContainerStarted","Data":"af36c5db6a4ea26709eed9d77dd458c91865e969444955b330b12b6d065ca672"} Jun 03 16:34:03.356372 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:03.356241 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" event={"ID":"b60d34f4-46cf-4530-91e1-7626ca6b4169","Type":"ContainerStarted","Data":"58be976574f0af803fcf268bc2d9a6781753c246fc46870e1cbf6e560d6259fb"} Jun 03 16:34:04.524925 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:04.524880 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:34:04.524925 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:04.524928 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:34:04.537500 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:04.537480 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:34:05.374059 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:05.374035 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27" Jun 03 16:34:09.382196 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:09.382165 2576 generic.go:358] "Generic (PLEG): container finished" podID="b60d34f4-46cf-4530-91e1-7626ca6b4169" containerID="af36c5db6a4ea26709eed9d77dd458c91865e969444955b330b12b6d065ca672" exitCode=0 Jun 03 16:34:09.382606 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:09.382248 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" event={"ID":"b60d34f4-46cf-4530-91e1-7626ca6b4169","Type":"ContainerDied","Data":"af36c5db6a4ea26709eed9d77dd458c91865e969444955b330b12b6d065ca672"} Jun 03 16:34:10.387452 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:10.387423 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" event={"ID":"b60d34f4-46cf-4530-91e1-7626ca6b4169","Type":"ContainerStarted","Data":"1b15904623037ffba950e2a537cc48a48d2b46250f6a185410379672d09f98bd"} Jun 03 16:34:10.437081 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:10.437023 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" podStartSLOduration=8.1413398 podStartE2EDuration="8.437005392s" podCreationTimestamp="2026-06-03 16:34:02 +0000 UTC" firstStartedPulling="2026-06-03 16:34:09.382851812 +0000 UTC m=+779.646915665" lastFinishedPulling="2026-06-03 16:34:09.678517394 +0000 UTC m=+779.942581257" observedRunningTime="2026-06-03 16:34:10.427229727 +0000 UTC m=+780.691293600" watchObservedRunningTime="2026-06-03 16:34:10.437005392 +0000 UTC m=+780.701069269" Jun 03 16:34:13.128401 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:13.128370 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:13.128401 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:13.128405 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:13.142845 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:13.142823 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:13.409528 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:13.409463 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv" Jun 03 16:34:35.926152 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:35.922560 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm"] Jun 03 16:34:35.930539 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:35.930514 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:35.933546 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:35.933520 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 03 16:34:35.940296 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:35.940269 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm"] Jun 03 16:34:36.119118 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.119084 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.119266 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.119128 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.119266 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.119218 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.119393 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.119289 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.119393 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.119311 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lm7w\" (UniqueName: \"kubernetes.io/projected/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-kube-api-access-7lm7w\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.119393 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.119360 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.220480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220409 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.220625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220483 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.220625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220503 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7lm7w\" (UniqueName: \"kubernetes.io/projected/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-kube-api-access-7lm7w\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.220625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220529 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.220625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220577 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.220625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220612 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.220910 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220846 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.220910 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220891 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.221012 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.220979 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.222854 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.222825 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.223140 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.223120 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.233720 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.233698 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lm7w\" (UniqueName: \"kubernetes.io/projected/2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec-kube-api-access-7lm7w\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm\" (UID: \"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.241577 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.241556 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" Jun 03 16:34:36.380295 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.380272 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm"] Jun 03 16:34:36.381702 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:34:36.381674 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2a1dc8_8ec2_42b7_a70f_e9a4909644ec.slice/crio-4d30a4292dc66828b3015ce535e6ddbb4c8551b9823e411370cbe44c94697a8b WatchSource:0}: Error finding container 4d30a4292dc66828b3015ce535e6ddbb4c8551b9823e411370cbe44c94697a8b: Status 404 returned error can't find the container with id 4d30a4292dc66828b3015ce535e6ddbb4c8551b9823e411370cbe44c94697a8b Jun 03 16:34:36.492193 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.492114 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" event={"ID":"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec","Type":"ContainerStarted","Data":"b1a7eb3aaefcf01f27ba5e782940db0b963996cfe47bd3a14febabc331e5aeff"} Jun 03 16:34:36.492193 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:34:36.492148 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm" event={"ID":"2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec","Type":"ContainerStarted","Data":"4d30a4292dc66828b3015ce535e6ddbb4c8551b9823e411370cbe44c94697a8b"} Jun 03 16:40:33.141480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:33.141400 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm_2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec/storage-initializer/0.log" Jun 03 16:40:34.080052 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:34.080018 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bf6964f95-vcsbx_0aaec692-9fa6-430b-8b8a-9c26f47172b9/authorino/0.log" Jun 03 16:40:38.599429 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:38.599400 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-5c5d698fd4-8tfkd_815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834/nginx/0.log" Jun 03 16:40:38.830341 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:38.830290 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-5tckm_db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1/manager/0.log" Jun 03 16:40:38.943584 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:38.943515 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-bdpsp_23fc4020-dd85-4e8f-a131-834db42b1559/manager/0.log" Jun 03 16:40:39.058018 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:39.057998 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6c84f6ccdc-7gp92_a55ef0f5-8a9d-410a-90f1-d55cffec34a2/maas-api/0.log" Jun 03 16:40:39.175303 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:39.175281 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f579569d9-nrpcv_399e7a68-b887-4d39-8cc9-9fdf79bcbc88/manager/0.log" Jun 03 16:40:39.285928 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:39.285907 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-gflc2_85e2770d-ae55-4f53-bac5-8f860dffb76b/server/0.log" Jun 03 16:40:39.402421 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:39.402404 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-spm6d_ae134d74-aebf-4fb2-a309-d5eaf3f813a2/manager/0.log" Jun 03 16:40:39.520492 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:39.520472 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84984b75f8-dnlsx_35147149-64fa-43cc-b747-cc40bbac566a/copy-manifests/0.log" Jun 03 16:40:39.527130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:39.527107 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84984b75f8-dnlsx_35147149-64fa-43cc-b747-cc40bbac566a/manager/0.log" Jun 03 16:40:40.659656 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.659627 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5_d80d6fd7-1bcc-4672-87d8-78c515faa376/util/0.log" Jun 03 16:40:40.665736 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.665716 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5_d80d6fd7-1bcc-4672-87d8-78c515faa376/pull/0.log" Jun 03 16:40:40.671968 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.671952 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5_d80d6fd7-1bcc-4672-87d8-78c515faa376/extract/0.log" Jun 03 16:40:40.783445 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.783422 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d_134b7fe6-30bc-41c7-9bc5-78d1bf07c12d/util/0.log" Jun 03 16:40:40.790247 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.790230 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d_134b7fe6-30bc-41c7-9bc5-78d1bf07c12d/pull/0.log" Jun 03 16:40:40.796529 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.796512 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d_134b7fe6-30bc-41c7-9bc5-78d1bf07c12d/extract/0.log" Jun 03 16:40:40.910232 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.910184 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc_ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595/util/0.log" Jun 03 16:40:40.916956 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.916937 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc_ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595/pull/0.log" Jun 03 16:40:40.923680 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:40.923661 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc_ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595/extract/0.log" Jun 03 16:40:41.029950 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:41.029930 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd_3526a40a-477a-45f8-b025-a0a2fcf3ce3e/util/0.log" Jun 03 16:40:41.035748 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:41.035730 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd_3526a40a-477a-45f8-b025-a0a2fcf3ce3e/pull/0.log" Jun 03 16:40:41.041190 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:41.041171 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd_3526a40a-477a-45f8-b025-a0a2fcf3ce3e/extract/0.log" Jun 03 16:40:41.153793 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:41.153778 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bf6964f95-vcsbx_0aaec692-9fa6-430b-8b8a-9c26f47172b9/authorino/0.log" Jun 03 16:40:41.281791 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:41.281768 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-rgz8v_b77ec22e-1b68-426a-a118-746866a3bf8c/manager/0.log" Jun 03 16:40:41.757548 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:41.757507 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-5kcsx_e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3/manager/0.log" Jun 03 16:40:42.465824 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:42.465796 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-qvwz8_57db75db-7542-4bc3-87e0-1367439c8160/discovery/0.log" Jun 03 16:40:42.692296 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:42.692271 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-574ffd5976-wdnwl_b775aa44-76cd-4941-9aff-8e54b4594a62/kube-auth-proxy/0.log" Jun 03 16:40:42.917173 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:42.917147 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-dhnd2_49cc4104-4b08-49d7-a025-ef3c6c440a28/payload-processing/0.log" Jun 03 16:40:43.368154 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:43.368130 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27_406c3b4d-d7e0-4f14-8865-3afc17f5ff99/storage-initializer/0.log" Jun 03 16:40:43.374315 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:43.374296 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-76x27_406c3b4d-d7e0-4f14-8865-3afc17f5ff99/main/0.log" Jun 03 16:40:43.603387 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:43.603363 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv_b60d34f4-46cf-4530-91e1-7626ca6b4169/storage-initializer/0.log" Jun 03 16:40:43.610237 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:43.610217 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-9k5vv_b60d34f4-46cf-4530-91e1-7626ca6b4169/main/0.log" Jun 03 16:40:43.962158 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:43.962129 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-98qjm_2a2a1dc8-8ec2-42b7-a70f-e9a4909644ec/storage-initializer/0.log" Jun 03 16:40:51.425217 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:51.425187 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-djvpx_2ebd340a-d9e9-41b0-9fe9-b70951a03707/global-pull-secret-syncer/0.log" Jun 03 16:40:51.654147 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:51.654121 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j5x6b_50816dc4-c2f9-4346-b8fa-d8e493440d5b/konnectivity-agent/0.log" Jun 03 16:40:51.718232 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:51.718158 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-141.ec2.internal_167b61c3f33681bb1d2fd6165981fb2f/haproxy/0.log" Jun 03 16:40:55.560137 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.560109 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5_d80d6fd7-1bcc-4672-87d8-78c515faa376/extract/0.log" Jun 03 16:40:55.582921 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.582896 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5_d80d6fd7-1bcc-4672-87d8-78c515faa376/util/0.log" Jun 03 16:40:55.604786 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.604764 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tcq5_d80d6fd7-1bcc-4672-87d8-78c515faa376/pull/0.log" Jun 03 16:40:55.629462 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.629441 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d_134b7fe6-30bc-41c7-9bc5-78d1bf07c12d/extract/0.log" Jun 03 16:40:55.652867 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.652847 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d_134b7fe6-30bc-41c7-9bc5-78d1bf07c12d/util/0.log" Jun 03 16:40:55.673765 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.673747 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q9w2d_134b7fe6-30bc-41c7-9bc5-78d1bf07c12d/pull/0.log" Jun 03 16:40:55.700997 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.700976 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc_ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595/extract/0.log" Jun 03 16:40:55.730166 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.730122 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc_ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595/util/0.log" Jun 03 16:40:55.750730 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.750713 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rpzvc_ee5a9e07-f2d2-4d8b-a73e-64b82b2c9595/pull/0.log" Jun 03 16:40:55.776261 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.776244 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd_3526a40a-477a-45f8-b025-a0a2fcf3ce3e/extract/0.log" Jun 03 16:40:55.799941 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.799921 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd_3526a40a-477a-45f8-b025-a0a2fcf3ce3e/util/0.log" Jun 03 16:40:55.824230 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.824181 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1j6rpd_3526a40a-477a-45f8-b025-a0a2fcf3ce3e/pull/0.log" Jun 03 16:40:55.855083 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.855063 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bf6964f95-vcsbx_0aaec692-9fa6-430b-8b8a-9c26f47172b9/authorino/0.log" Jun 03 16:40:55.880213 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:55.880192 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-rgz8v_b77ec22e-1b68-426a-a118-746866a3bf8c/manager/0.log" Jun 03 16:40:56.005505 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:56.005482 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-5kcsx_e5d3eb2e-e2a8-413d-ab03-2cb5da15dee3/manager/0.log" Jun 03 16:40:57.497925 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.497899 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-sppp4_2d202eb5-629e-464a-bd86-e7810e466d94/kube-state-metrics/0.log" Jun 03 16:40:57.520875 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.520854 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-sppp4_2d202eb5-629e-464a-bd86-e7810e466d94/kube-rbac-proxy-main/0.log" Jun 03 16:40:57.543116 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.543092 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-sppp4_2d202eb5-629e-464a-bd86-e7810e466d94/kube-rbac-proxy-self/0.log" Jun 03 16:40:57.570654 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.570638 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c58d6945f-4bc4b_7abb010c-e3ca-4235-8ac5-b0c6c501b813/metrics-server/0.log" Jun 03 16:40:57.593164 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.593145 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-8ddxl_6712278d-f593-49fb-926e-30788a6a7194/monitoring-plugin/0.log" Jun 03 16:40:57.757020 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.756999 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nt5ph_c3a97e66-c7f4-4b85-b7d7-590ef5b3321b/node-exporter/0.log" Jun 03 16:40:57.783246 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.783228 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nt5ph_c3a97e66-c7f4-4b85-b7d7-590ef5b3321b/kube-rbac-proxy/0.log" Jun 03 16:40:57.805696 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.805680 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nt5ph_c3a97e66-c7f4-4b85-b7d7-590ef5b3321b/init-textfile/0.log" Jun 03 16:40:57.835999 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.835980 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-xlzcd_756ce28d-9e95-445e-a661-11f9b38dd228/kube-rbac-proxy-main/0.log" Jun 03 16:40:57.856611 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.856591 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-xlzcd_756ce28d-9e95-445e-a661-11f9b38dd228/kube-rbac-proxy-self/0.log" Jun 03 16:40:57.878800 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:57.878780 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-xlzcd_756ce28d-9e95-445e-a661-11f9b38dd228/openshift-state-metrics/0.log" Jun 03 16:40:58.083268 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.083201 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-566kj_8b29281c-7554-4815-a6d7-2f509a99dc6e/prometheus-operator/0.log" Jun 03 16:40:58.106480 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.106456 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-566kj_8b29281c-7554-4815-a6d7-2f509a99dc6e/kube-rbac-proxy/0.log" Jun 03 16:40:58.155332 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.155304 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-zb2kr_dcfff5a3-832d-4b05-a0f0-acf690d13419/prometheus-operator-admission-webhook/0.log" Jun 03 16:40:58.204242 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.204217 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77d48fbc65-nkxr5_bb2d28b8-a41f-4014-a23d-cce589335ff3/telemeter-client/0.log" Jun 03 16:40:58.230194 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.230132 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77d48fbc65-nkxr5_bb2d28b8-a41f-4014-a23d-cce589335ff3/reload/0.log" Jun 03 16:40:58.262025 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.262005 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77d48fbc65-nkxr5_bb2d28b8-a41f-4014-a23d-cce589335ff3/kube-rbac-proxy/0.log" Jun 03 16:40:58.295670 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.295650 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b4669468d-hvr8h_35443b97-64f9-4797-bb47-81eebf92b93a/thanos-query/0.log" Jun 03 16:40:58.323810 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.323794 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b4669468d-hvr8h_35443b97-64f9-4797-bb47-81eebf92b93a/kube-rbac-proxy-web/0.log" Jun 03 16:40:58.344829 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.344772 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b4669468d-hvr8h_35443b97-64f9-4797-bb47-81eebf92b93a/kube-rbac-proxy/0.log" Jun 03 16:40:58.365798 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.365777 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b4669468d-hvr8h_35443b97-64f9-4797-bb47-81eebf92b93a/prom-label-proxy/0.log" Jun 03 16:40:58.384739 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.384721 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b4669468d-hvr8h_35443b97-64f9-4797-bb47-81eebf92b93a/kube-rbac-proxy-rules/0.log" Jun 03 16:40:58.405416 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:40:58.405397 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b4669468d-hvr8h_35443b97-64f9-4797-bb47-81eebf92b93a/kube-rbac-proxy-metrics/0.log" Jun 03 16:41:00.602396 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:00.602371 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-645b8bb455-8kknv_5ebfaa11-f736-498d-9573-9d028b5f0f82/console/0.log" Jun 03 16:41:00.958563 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:00.958469 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl"] Jun 03 16:41:00.962082 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:00.962065 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:00.964662 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:00.964646 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sp5fs\"/\"kube-root-ca.crt\"" Jun 03 16:41:00.964739 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:00.964648 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sp5fs\"/\"openshift-service-ca.crt\"" Jun 03 16:41:00.965596 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:00.965579 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sp5fs\"/\"default-dockercfg-9ssjr\"" Jun 03 16:41:00.971392 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:00.971371 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl"] Jun 03 16:41:01.098006 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.097978 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvwhx\" (UniqueName: \"kubernetes.io/projected/3c6c0b8a-738a-4987-8287-71442b5d68e8-kube-api-access-wvwhx\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.098164 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.098051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-podres\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.098164 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.098086 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-proc\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.098164 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.098101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-sys\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.098164 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.098118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-lib-modules\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.198789 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198761 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-podres\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.198926 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198805 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-proc\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.198926 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198824 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-sys\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.198926 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198849 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-lib-modules\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.198926 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198884 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wvwhx\" (UniqueName: \"kubernetes.io/projected/3c6c0b8a-738a-4987-8287-71442b5d68e8-kube-api-access-wvwhx\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.198926 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198895 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-proc\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.198926 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198914 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-sys\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.199139 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198920 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-podres\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.199139 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.198964 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c6c0b8a-738a-4987-8287-71442b5d68e8-lib-modules\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.206625 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.206606 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvwhx\" (UniqueName: \"kubernetes.io/projected/3c6c0b8a-738a-4987-8287-71442b5d68e8-kube-api-access-wvwhx\") pod \"perf-node-gather-daemonset-bwxjl\" (UID: \"3c6c0b8a-738a-4987-8287-71442b5d68e8\") " pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.272778 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.272757 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:01.607899 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.607873 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl"] Jun 03 16:41:01.608831 ip-10-0-128-141 kubenswrapper[2576]: W0603 16:41:01.608803 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3c6c0b8a_738a_4987_8287_71442b5d68e8.slice/crio-a09fac17f5a0aa2a2f78a869049e0216a8f2d9193ae880f8525e49f36132dfab WatchSource:0}: Error finding container a09fac17f5a0aa2a2f78a869049e0216a8f2d9193ae880f8525e49f36132dfab: Status 404 returned error can't find the container with id a09fac17f5a0aa2a2f78a869049e0216a8f2d9193ae880f8525e49f36132dfab Jun 03 16:41:01.610413 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:01.610397 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:41:02.035270 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:02.035241 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-59bdh_e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d/dns/0.log" Jun 03 16:41:02.057519 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:02.057499 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-59bdh_e8ff0c4e-d93c-47b4-ac77-8bc640d4aa6d/kube-rbac-proxy/0.log" Jun 03 16:41:02.093230 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:02.093207 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" event={"ID":"3c6c0b8a-738a-4987-8287-71442b5d68e8","Type":"ContainerStarted","Data":"362d939d57ac8fbcd58622391b387f104c934309d229e3382b372c39cda49b71"} Jun 03 16:41:02.093363 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:02.093239 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" event={"ID":"3c6c0b8a-738a-4987-8287-71442b5d68e8","Type":"ContainerStarted","Data":"a09fac17f5a0aa2a2f78a869049e0216a8f2d9193ae880f8525e49f36132dfab"} Jun 03 16:41:02.093363 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:02.093254 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:02.113038 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:02.113000 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" podStartSLOduration=2.112988399 podStartE2EDuration="2.112988399s" podCreationTimestamp="2026-06-03 16:41:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:41:02.111663882 +0000 UTC m=+1192.375727757" watchObservedRunningTime="2026-06-03 16:41:02.112988399 +0000 UTC m=+1192.377052273" Jun 03 16:41:02.205130 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:02.205106 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pndzh_9e067270-9c93-492a-8247-37165df0c87d/dns-node-resolver/0.log" Jun 03 16:41:02.736366 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:02.736342 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v7vzl_8d268c08-9ed4-4065-bb9a-59e46edb4820/node-ca/0.log" Jun 03 16:41:03.571468 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:03.571438 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-qvwz8_57db75db-7542-4bc3-87e0-1367439c8160/discovery/0.log" Jun 03 16:41:03.611906 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:03.611882 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-574ffd5976-wdnwl_b775aa44-76cd-4941-9aff-8e54b4594a62/kube-auth-proxy/0.log" Jun 03 16:41:03.661299 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:03.661278 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-dhnd2_49cc4104-4b08-49d7-a025-ef3c6c440a28/payload-processing/0.log" Jun 03 16:41:04.222306 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:04.222276 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m458w_35619cd5-0a7d-4752-bbc3-321dd5e1b1da/serve-healthcheck-canary/0.log" Jun 03 16:41:04.757718 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:04.757692 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-scd2b_4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8/kube-rbac-proxy/0.log" Jun 03 16:41:04.776570 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:04.776548 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-scd2b_4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8/exporter/0.log" Jun 03 16:41:04.794680 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:04.794647 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-scd2b_4f5834ad-1a0b-4dd4-a05b-c4680b2bb5e8/extractor/0.log" Jun 03 16:41:06.776070 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.776040 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-5c5d698fd4-8tfkd_815ec6f4-ff7c-4e1a-8576-3ea9bcf6c834/nginx/0.log" Jun 03 16:41:06.818004 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.817980 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-5tckm_db2c46c0-3e4f-4ac5-80bd-f660ccf5dbb1/manager/0.log" Jun 03 16:41:06.848684 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.848660 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-bdpsp_23fc4020-dd85-4e8f-a131-834db42b1559/manager/0.log" Jun 03 16:41:06.867815 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.867793 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6c84f6ccdc-7gp92_a55ef0f5-8a9d-410a-90f1-d55cffec34a2/maas-api/0.log" Jun 03 16:41:06.890833 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.890808 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f579569d9-nrpcv_399e7a68-b887-4d39-8cc9-9fdf79bcbc88/manager/0.log" Jun 03 16:41:06.907967 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.907944 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-gflc2_85e2770d-ae55-4f53-bac5-8f860dffb76b/server/0.log" Jun 03 16:41:06.953203 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.953186 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-spm6d_ae134d74-aebf-4fb2-a309-d5eaf3f813a2/manager/0.log" Jun 03 16:41:06.975150 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.975129 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84984b75f8-dnlsx_35147149-64fa-43cc-b747-cc40bbac566a/manager/0.log" Jun 03 16:41:06.994198 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:06.994180 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84984b75f8-dnlsx_35147149-64fa-43cc-b747-cc40bbac566a/copy-manifests/0.log" Jun 03 16:41:08.106288 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:08.106263 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sp5fs/perf-node-gather-daemonset-bwxjl" Jun 03 16:41:14.723466 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:14.723437 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jjcb8_fc8c28fa-c3a1-4e56-a531-e470dec2faca/kube-multus-additional-cni-plugins/0.log" Jun 03 16:41:14.745807 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:14.745763 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jjcb8_fc8c28fa-c3a1-4e56-a531-e470dec2faca/egress-router-binary-copy/0.log" Jun 03 16:41:14.766333 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:14.766298 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jjcb8_fc8c28fa-c3a1-4e56-a531-e470dec2faca/cni-plugins/0.log" Jun 03 16:41:14.790661 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:14.790640 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jjcb8_fc8c28fa-c3a1-4e56-a531-e470dec2faca/bond-cni-plugin/0.log" Jun 03 16:41:14.809734 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:14.809717 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jjcb8_fc8c28fa-c3a1-4e56-a531-e470dec2faca/routeoverride-cni/0.log" Jun 03 16:41:14.828354 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:14.828314 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jjcb8_fc8c28fa-c3a1-4e56-a531-e470dec2faca/whereabouts-cni-bincopy/0.log" Jun 03 16:41:14.849266 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:14.849251 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jjcb8_fc8c28fa-c3a1-4e56-a531-e470dec2faca/whereabouts-cni/0.log" Jun 03 16:41:15.044855 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:15.044831 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cmrwn_9a7e81d0-b84a-4301-a034-17a0005c76cb/kube-multus/0.log" Jun 03 16:41:15.121231 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:15.121206 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-89gzm_06952f51-6375-47cd-b827-88ef7b5d38ae/network-metrics-daemon/0.log" Jun 03 16:41:15.139040 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:15.139002 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-89gzm_06952f51-6375-47cd-b827-88ef7b5d38ae/kube-rbac-proxy/0.log" Jun 03 16:41:16.274002 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:16.273967 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68jfx_2b150971-2aa3-4738-948a-256e18c8d0ef/ovn-controller/0.log" Jun 03 16:41:16.293010 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:16.292987 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68jfx_2b150971-2aa3-4738-948a-256e18c8d0ef/ovn-acl-logging/0.log" Jun 03 16:41:16.308632 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:16.308612 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68jfx_2b150971-2aa3-4738-948a-256e18c8d0ef/kube-rbac-proxy-node/0.log" Jun 03 16:41:16.327097 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:16.327074 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68jfx_2b150971-2aa3-4738-948a-256e18c8d0ef/kube-rbac-proxy-ovn-metrics/0.log" Jun 03 16:41:16.345214 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:16.345199 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68jfx_2b150971-2aa3-4738-948a-256e18c8d0ef/northd/0.log" Jun 03 16:41:16.364066 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:16.364048 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68jfx_2b150971-2aa3-4738-948a-256e18c8d0ef/nbdb/0.log" Jun 03 16:41:16.383161 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:16.383141 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68jfx_2b150971-2aa3-4738-948a-256e18c8d0ef/sbdb/0.log" Jun 03 16:41:16.483546 ip-10-0-128-141 kubenswrapper[2576]: I0603 16:41:16.483524 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68jfx_2b150971-2aa3-4738-948a-256e18c8d0ef/ovnkube-controller/0.log"