Jun 08 18:12:36.548856 ip-10-0-131-227 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 08 18:12:36.548871 ip-10-0-131-227 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 08 18:12:36.548879 ip-10-0-131-227 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 08 18:12:36.549217 ip-10-0-131-227 systemd[1]: Failed to start Kubernetes Kubelet. Jun 08 18:12:46.672170 ip-10-0-131-227 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 08 18:12:46.672193 ip-10-0-131-227 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c58feaafedc8411798af7e9254a12c1d -- Jun 08 18:15:25.718123 ip-10-0-131-227 systemd[1]: Starting Kubernetes Kubelet... Jun 08 18:15:26.152630 ip-10-0-131-227 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 18:15:26.152630 ip-10-0-131-227 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 08 18:15:26.152630 ip-10-0-131-227 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 18:15:26.152630 ip-10-0-131-227 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 08 18:15:26.152630 ip-10-0-131-227 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 18:15:26.154990 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.154908 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 08 18:15:26.160616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160602 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:15:26.160616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160616 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160620 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160625 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160628 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160631 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160634 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160637 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160639 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160642 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160645 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160648 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160650 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160652 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160655 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160658 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160660 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160663 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160666 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160668 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160671 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:15:26.160679 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160681 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160684 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160687 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160690 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160692 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160695 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160697 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160700 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160702 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160705 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160707 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160709 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160712 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160714 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160717 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160719 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160722 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160725 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160727 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160730 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:15:26.161224 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160733 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160736 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160738 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160741 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160743 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160746 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160750 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160752 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160755 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160757 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160760 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160763 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160765 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160768 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160771 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160775 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160779 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160782 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160785 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:15:26.162065 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160788 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160790 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160793 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160795 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160798 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160800 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160803 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160805 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160808 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160811 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160813 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160816 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160819 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160821 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160824 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160826 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160828 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160831 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160834 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160836 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:15:26.162568 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160839 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160859 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160865 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160868 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160871 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160874 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160876 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160880 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160883 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160886 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160889 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160891 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160895 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160898 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160901 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160904 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160907 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160910 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:15:26.163071 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.160913 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:15:26.164318 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164296 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:15:26.164318 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164318 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164322 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164325 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164328 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164331 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164333 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164336 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164338 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164341 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164343 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164347 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164350 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164352 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164355 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164357 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164360 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164362 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164365 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164367 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164370 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:15:26.164382 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164372 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164375 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164377 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164380 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164382 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164385 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164391 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164394 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164397 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164399 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164402 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164405 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164407 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164410 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164413 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164415 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164418 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164422 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164426 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:15:26.164886 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164429 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164432 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164435 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164439 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164443 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164445 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164448 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164451 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164453 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164456 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164459 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164462 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164464 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164467 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164469 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164473 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164476 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164478 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164481 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164484 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:15:26.165354 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164486 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164489 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164491 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164495 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164500 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164503 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164505 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164508 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164511 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164514 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164517 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164520 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164522 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164525 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164528 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164530 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164533 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164535 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164538 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:15:26.165860 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164540 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164543 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164545 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164548 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164551 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164553 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164557 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164559 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164562 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164565 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164568 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164570 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164573 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164575 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164578 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164580 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164583 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164585 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164588 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:15:26.166319 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164591 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164982 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164988 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164991 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164994 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164997 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.164999 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165002 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165005 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165007 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165010 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165013 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165020 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165022 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165025 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165028 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165030 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165034 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165037 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165039 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:15:26.166785 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165042 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165045 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165048 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165050 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165053 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165055 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165058 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165060 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165063 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165065 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165068 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165071 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165073 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165076 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165078 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165081 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165084 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165086 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165089 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165091 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:15:26.167291 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165094 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165096 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165099 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165101 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165104 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165106 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165109 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165111 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165114 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165116 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165120 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165123 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165125 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165128 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165131 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165133 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165136 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165138 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165140 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165143 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:15:26.167782 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165146 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165148 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165150 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165153 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165156 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165163 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165166 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165168 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165171 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165173 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165176 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165178 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165181 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165183 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165186 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165188 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165195 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165198 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165201 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:15:26.168283 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165204 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165207 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165210 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165213 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165216 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165218 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165221 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165224 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165226 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165229 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165232 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165235 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165238 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165240 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165243 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165246 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165250 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165253 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165256 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:15:26.168752 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165259 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165262 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165338 2560 flags.go:64] FLAG: --address="0.0.0.0" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165345 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165351 2560 flags.go:64] FLAG: --anonymous-auth="true" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165356 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165360 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165363 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165367 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165371 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165376 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165378 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165382 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165385 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165388 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165391 2560 flags.go:64] FLAG: --cgroup-root="" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165393 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165396 2560 flags.go:64] FLAG: --client-ca-file="" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165399 2560 flags.go:64] FLAG: --cloud-provider="external" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165403 2560 flags.go:64] FLAG: --cluster-dns="[]" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165408 2560 flags.go:64] FLAG: --cluster-domain="" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165410 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165413 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 08 18:15:26.169267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165416 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165419 2560 flags.go:64] FLAG: --container-log-max-files="5" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165427 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165430 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165433 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165436 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165439 2560 flags.go:64] FLAG: --contention-profiling="false" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165442 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165445 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165448 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165451 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165454 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165457 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165460 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165463 2560 flags.go:64] FLAG: --enable-load-reader="false" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165465 2560 flags.go:64] FLAG: --enable-server="true" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165468 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165473 2560 flags.go:64] FLAG: --event-burst="100" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165476 2560 flags.go:64] FLAG: --event-qps="50" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165480 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165483 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165486 2560 flags.go:64] FLAG: --eviction-hard="" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165490 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165493 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165496 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 08 18:15:26.169830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165499 2560 flags.go:64] FLAG: --eviction-soft="" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165502 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165504 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165513 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165516 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165518 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165521 2560 flags.go:64] FLAG: --fail-swap-on="true" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165524 2560 flags.go:64] FLAG: --feature-gates="" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165528 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165531 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165534 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165537 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165540 2560 flags.go:64] FLAG: --healthz-port="10248" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165542 2560 flags.go:64] FLAG: --help="false" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165545 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165548 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165551 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165553 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165557 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165560 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165562 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165565 2560 flags.go:64] FLAG: --image-service-endpoint="" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165568 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 08 18:15:26.170455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165570 2560 flags.go:64] FLAG: --kube-api-burst="100" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165573 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165576 2560 flags.go:64] FLAG: --kube-api-qps="50" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165580 2560 flags.go:64] FLAG: --kube-reserved="" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165582 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165585 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165588 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165591 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165593 2560 flags.go:64] FLAG: --lock-file="" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165596 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165599 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165602 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165608 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165611 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165614 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165617 2560 flags.go:64] FLAG: --logging-format="text" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165619 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165623 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165625 2560 flags.go:64] FLAG: --manifest-url="" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165628 2560 flags.go:64] FLAG: --manifest-url-header="" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165632 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165635 2560 flags.go:64] FLAG: --max-open-files="1000000" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165639 2560 flags.go:64] FLAG: --max-pods="110" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165642 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165645 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 08 18:15:26.171048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165647 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165650 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165653 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165656 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165658 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165665 2560 flags.go:64] FLAG: --node-status-max-images="50" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165668 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165671 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165673 2560 flags.go:64] FLAG: --pod-cidr="" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165676 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165681 2560 flags.go:64] FLAG: --pod-manifest-path="" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165685 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165689 2560 flags.go:64] FLAG: --pods-per-core="0" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165691 2560 flags.go:64] FLAG: --port="10250" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165694 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165697 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-027e5b5416b10b788" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165700 2560 flags.go:64] FLAG: --qos-reserved="" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165703 2560 flags.go:64] FLAG: --read-only-port="10255" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165706 2560 flags.go:64] FLAG: --register-node="true" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165710 2560 flags.go:64] FLAG: --register-with-taints="" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165713 2560 flags.go:64] FLAG: --registry-burst="10" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165716 2560 flags.go:64] FLAG: --registry-qps="5" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165719 2560 flags.go:64] FLAG: --reserved-cpus="" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165722 2560 flags.go:64] FLAG: --reserved-memory="" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165728 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 08 18:15:26.171673 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165731 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165733 2560 flags.go:64] FLAG: --rotate-certificates="false" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165737 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165739 2560 flags.go:64] FLAG: --runonce="false" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165742 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165745 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165748 2560 flags.go:64] FLAG: --seccomp-default="false" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165750 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165753 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165756 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165758 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165761 2560 flags.go:64] FLAG: --storage-driver-password="root" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165764 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165767 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165769 2560 flags.go:64] FLAG: --storage-driver-user="root" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165772 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165775 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165778 2560 flags.go:64] FLAG: --system-cgroups="" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165781 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165786 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165789 2560 flags.go:64] FLAG: --tls-cert-file="" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165792 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165796 2560 flags.go:64] FLAG: --tls-min-version="" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165798 2560 flags.go:64] FLAG: --tls-private-key-file="" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165801 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jun 08 18:15:26.172335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165804 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165807 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165810 2560 flags.go:64] FLAG: --v="2" Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165814 2560 flags.go:64] FLAG: --version="false" Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165820 2560 flags.go:64] FLAG: --vmodule="" Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165824 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.165827 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165982 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165988 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165991 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165994 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165997 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.165999 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166002 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166005 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166007 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166010 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166012 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166015 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166017 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166020 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166022 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:15:26.172947 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166025 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166028 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166031 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166036 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166038 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166041 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166043 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166046 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166048 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166051 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166053 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166058 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166060 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166063 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166065 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166072 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166075 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166077 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166080 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:15:26.173486 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166082 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166085 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166087 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166090 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166092 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166094 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166097 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166100 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166102 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166105 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166110 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166114 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166117 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166120 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166123 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166126 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166130 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166134 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166138 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:15:26.173971 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166141 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166143 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166146 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166148 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166151 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166155 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166157 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166160 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166162 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166164 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166168 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166170 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166173 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166175 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166177 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166180 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166182 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166185 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166187 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166190 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:15:26.174469 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166193 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166197 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166199 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166202 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166204 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166207 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166210 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166212 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166215 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166218 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166221 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166224 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166226 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166229 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166232 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166234 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166236 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166240 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166243 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166245 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:15:26.174975 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166248 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166250 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166253 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166256 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166258 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.166261 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.166266 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.173162 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.173270 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.173290 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.173296 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173350 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173354 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173358 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:15:26.175458 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173361 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173363 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173366 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173369 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173371 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173374 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173376 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173379 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173382 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173384 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173387 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173390 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173392 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173395 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173397 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173401 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173403 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173406 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173408 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173411 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173414 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:15:26.175856 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173417 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173420 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173422 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173425 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173427 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173430 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173433 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173435 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173438 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173440 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173443 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173445 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173448 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173450 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173452 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173455 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173458 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173460 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173463 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173466 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:15:26.176385 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173468 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173471 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173473 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173476 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173478 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173481 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173484 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173486 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173490 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173494 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173497 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173500 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173503 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173506 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173508 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173512 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173516 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173519 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173522 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:15:26.176911 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173525 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173528 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173530 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173533 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173536 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173538 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173541 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173544 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173546 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173549 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173551 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173554 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173557 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173559 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173561 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173564 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173566 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173569 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173571 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173574 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:15:26.177366 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173576 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173580 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173585 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173587 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173590 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173592 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173595 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173597 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173600 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173602 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173605 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173607 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173610 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173612 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173615 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:15:26.177890 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173618 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.173623 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173731 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173735 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173738 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173741 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173743 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173746 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173749 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173751 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173754 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173756 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173759 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173761 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:15:26.178256 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173764 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173766 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173770 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173774 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173776 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173779 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173781 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173784 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173786 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173789 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173791 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173793 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173796 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173798 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173801 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173803 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173806 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173808 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173811 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:15:26.178594 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173814 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173818 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173821 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173824 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173827 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173829 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173832 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173835 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173837 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173857 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173861 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173864 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173867 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173870 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173873 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173876 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173879 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173882 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173884 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173887 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:15:26.179128 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173890 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173893 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173895 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173898 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173900 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173903 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173905 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173908 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173910 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173913 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173916 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173919 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173921 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173924 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173926 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173929 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173931 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173934 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173936 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173939 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:15:26.179616 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173941 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173943 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173946 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173948 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173951 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173954 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173956 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173958 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173961 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173964 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173966 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173970 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173973 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173976 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173979 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173982 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173985 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173988 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173990 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:15:26.180105 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173993 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173995 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.173998 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.174000 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.174004 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.174006 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.174008 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.174011 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:26.174013 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.174018 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.174123 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.176243 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.177555 2560 server.go:1019] "Starting client certificate rotation" Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.177650 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 18:15:26.180554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.177719 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 18:15:26.181332 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.181319 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 18:15:26.196896 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.196871 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 18:15:26.210118 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.210098 2560 log.go:25] "Validated CRI v1 runtime API" Jun 08 18:15:26.214188 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.214171 2560 log.go:25] "Validated CRI v1 image API" Jun 08 18:15:26.214995 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.214980 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 08 18:15:26.217880 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.217823 2560 fs.go:135] Filesystem UUIDs: map[0ee08641-a043-4f21-80bd-716c454e83cc:/dev/nvme0n1p4 2359a0d4-9fbb-46d3-bdd4-6627527ec94d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jun 08 18:15:26.217880 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.217873 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 08 18:15:26.221144 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.221115 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 18:15:26.223227 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.223117 2560 manager.go:216] Machine: {Timestamp:2026-06-08 18:15:26.221884959 +0000 UTC m=+0.382721607 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095386 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2327a2dea1754c23b491c5e23bf0f8 SystemUUID:ec2327a2-dea1-754c-23b4-91c5e23bf0f8 BootID:c58feaaf-edc8-4117-98af-7e9254a12c1d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:75:58:88:81:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:75:58:88:81:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:5a:84:8a:29:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 08 18:15:26.223227 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.223219 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 08 18:15:26.223341 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.223332 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 08 18:15:26.226888 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.226858 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 08 18:15:26.227019 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.226892 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-227.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 08 18:15:26.227619 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.227609 2560 topology_manager.go:138] "Creating topology manager with none policy" Jun 08 18:15:26.227652 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.227621 2560 container_manager_linux.go:309] "Creating device plugin manager" Jun 08 18:15:26.227652 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.227629 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 18:15:26.228437 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.228428 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 18:15:26.229331 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.229321 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 08 18:15:26.232489 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232477 2560 state_mem.go:36] "Initialized new in-memory state store" Jun 08 18:15:26.232587 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232577 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 08 18:15:26.232619 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232605 2560 kubelet.go:486] "Attempting to sync node with API server" Jun 08 18:15:26.232619 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232615 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 08 18:15:26.232669 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232631 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 08 18:15:26.232669 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232641 2560 kubelet.go:398] "Adding apiserver pod source" Jun 08 18:15:26.232669 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232648 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 08 18:15:26.232991 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232979 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 18:15:26.233038 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.232999 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 18:15:26.243999 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.243967 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ql9b2" Jun 08 18:15:26.245178 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.245159 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 08 18:15:26.246922 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.246909 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 08 18:15:26.246970 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.246939 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 08 18:15:26.248387 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248375 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 08 18:15:26.248434 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248392 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 08 18:15:26.248434 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248398 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 08 18:15:26.248434 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248403 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 08 18:15:26.248434 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248413 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 08 18:15:26.248434 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248422 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 08 18:15:26.248434 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248434 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 08 18:15:26.248594 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248440 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 08 18:15:26.248594 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248446 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 08 18:15:26.248594 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248452 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 08 18:15:26.248594 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248460 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 08 18:15:26.248594 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.248467 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 08 18:15:26.249449 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.249439 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 08 18:15:26.249449 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.249449 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 08 18:15:26.250177 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.249839 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 08 18:15:26.250380 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.250352 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-227.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 08 18:15:26.253203 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.253173 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ql9b2" Jun 08 18:15:26.254964 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.254947 2560 server.go:1268] "Started kubelet" Jun 08 18:15:26.255073 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.255048 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 08 18:15:26.255173 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.255098 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 08 18:15:26.255260 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.255201 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 08 18:15:26.255887 ip-10-0-131-227 systemd[1]: Started Kubernetes Kubelet. Jun 08 18:15:26.257010 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.256989 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 08 18:15:26.257180 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.257168 2560 server.go:310] "Adding debug handlers to kubelet server" Jun 08 18:15:26.258642 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.258625 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-227.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 08 18:15:26.260771 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.260750 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 08 18:15:26.261553 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.261534 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 08 18:15:26.261626 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.261536 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 08 18:15:26.261678 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.261640 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:26.262100 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.262084 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 08 18:15:26.262239 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.262225 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 08 18:15:26.262305 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.262241 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 08 18:15:26.262353 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.262319 2560 reconstruct.go:97] "Volume reconstruction finished" Jun 08 18:15:26.262353 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.262324 2560 reconciler.go:29] "Reconciler: start to sync state" Jun 08 18:15:26.264519 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.264505 2560 factory.go:55] Registering systemd factory Jun 08 18:15:26.264519 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.264520 2560 factory.go:223] Registration of the systemd container factory successfully Jun 08 18:15:26.264743 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.264735 2560 factory.go:153] Registering CRI-O factory Jun 08 18:15:26.264743 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.264743 2560 factory.go:223] Registration of the crio container factory successfully Jun 08 18:15:26.264814 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.264781 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 08 18:15:26.264814 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.264803 2560 factory.go:103] Registering Raw factory Jun 08 18:15:26.264814 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.264813 2560 manager.go:1196] Started watching for new ooms in manager Jun 08 18:15:26.265223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.265207 2560 manager.go:349] Starting recovery of all containers Jun 08 18:15:26.268158 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.268140 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:15:26.272784 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.272764 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-227.ec2.internal\" not found" node="ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.277562 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.277549 2560 manager.go:354] Recovery completed Jun 08 18:15:26.282464 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.282452 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:15:26.284324 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.284303 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:15:26.284387 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.284342 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:15:26.284387 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.284357 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:15:26.284978 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.284963 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 08 18:15:26.284978 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.284976 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 08 18:15:26.285076 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.284984 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:15:26.285076 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.284992 2560 state_mem.go:36] "Initialized new in-memory state store" Jun 08 18:15:26.285646 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.285630 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:15:26.285716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.285655 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:15:26.285716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.285665 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:15:26.286390 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.286378 2560 policy_none.go:49] "None policy: Start" Jun 08 18:15:26.286431 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.286394 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 08 18:15:26.286431 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.286404 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 08 18:15:26.287521 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.287510 2560 policy_none.go:47] "Start" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.327398 2560 manager.go:337] "Starting Device Plugin manager" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.327464 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.327474 2560 server.go:87] "Starting device plugin registration server" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.327723 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.327734 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.327796 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.327907 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.327916 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.329101 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.329158 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.329637 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.331272 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.331311 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.331337 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.331387 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 08 18:15:26.335380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.333660 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:15:26.428602 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.428545 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:15:26.429414 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.429393 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:15:26.429486 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.429429 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:15:26.429486 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.429450 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:15:26.429486 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.429474 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.431486 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.431460 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal"] Jun 08 18:15:26.431576 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.431540 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:15:26.432339 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.432321 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:15:26.432402 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.432352 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:15:26.432402 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.432362 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:15:26.433608 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.433596 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:15:26.433773 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.433759 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.433830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.433801 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:15:26.435206 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.435190 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:15:26.435292 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.435222 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:15:26.435292 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.435231 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:15:26.435292 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.435230 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:15:26.435292 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.435260 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:15:26.435292 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.435275 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:15:26.436454 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.436442 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.436508 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.436465 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:15:26.437071 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.437058 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:15:26.437121 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.437079 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:15:26.437121 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.437094 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:15:26.439820 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.439806 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.439890 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.439825 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-227.ec2.internal\": node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:26.463810 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.463778 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05b840dcb7cfc31951c9194a12f1eb48-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"05b840dcb7cfc31951c9194a12f1eb48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.463943 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.463819 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0bdb72680f38d3f1fa35164d90fd6e0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-227.ec2.internal\" (UID: \"b0bdb72680f38d3f1fa35164d90fd6e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.463943 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.463865 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05b840dcb7cfc31951c9194a12f1eb48-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"05b840dcb7cfc31951c9194a12f1eb48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.465395 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.465376 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:26.470589 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.470566 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-227.ec2.internal\" not found" node="ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.475006 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.474990 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-227.ec2.internal\" not found" node="ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.564249 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.564225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05b840dcb7cfc31951c9194a12f1eb48-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"05b840dcb7cfc31951c9194a12f1eb48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.564329 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.564250 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0bdb72680f38d3f1fa35164d90fd6e0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-227.ec2.internal\" (UID: \"b0bdb72680f38d3f1fa35164d90fd6e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.564329 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.564265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05b840dcb7cfc31951c9194a12f1eb48-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"05b840dcb7cfc31951c9194a12f1eb48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.564329 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.564299 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05b840dcb7cfc31951c9194a12f1eb48-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"05b840dcb7cfc31951c9194a12f1eb48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.564329 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.564322 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05b840dcb7cfc31951c9194a12f1eb48-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"05b840dcb7cfc31951c9194a12f1eb48\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.564463 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.564325 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0bdb72680f38d3f1fa35164d90fd6e0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-227.ec2.internal\" (UID: \"b0bdb72680f38d3f1fa35164d90fd6e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.566359 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.566333 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:26.667165 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.667138 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:26.767417 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.767354 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:26.775477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.775441 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.779025 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:26.779007 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jun 08 18:15:26.868404 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.868369 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:26.968904 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:26.968875 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:27.069491 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:27.069420 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:27.084982 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.084946 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:15:27.169731 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:27.169695 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:27.176871 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.176833 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 08 18:15:27.177026 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.177004 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 18:15:27.177092 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.177004 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 18:15:27.177092 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.177008 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 18:15:27.259413 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.259364 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-07 18:10:26 +0000 UTC" deadline="2028-02-09 23:05:30.851207222 +0000 UTC" Jun 08 18:15:27.259413 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.259406 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14668h50m3.591805586s" Jun 08 18:15:27.270645 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:27.270612 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:27.323809 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:27.323631 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b840dcb7cfc31951c9194a12f1eb48.slice/crio-b4303c4e099e0487b4eaee2c32acb2c94e0b50167106bcc24917eff9a506032b WatchSource:0}: Error finding container b4303c4e099e0487b4eaee2c32acb2c94e0b50167106bcc24917eff9a506032b: Status 404 returned error can't find the container with id b4303c4e099e0487b4eaee2c32acb2c94e0b50167106bcc24917eff9a506032b Jun 08 18:15:27.323998 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:27.323979 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0bdb72680f38d3f1fa35164d90fd6e0.slice/crio-da1c1a8db1724eaba0187e06f77a14f43a7f1dc7db60d8828ad9b066cf99532d WatchSource:0}: Error finding container da1c1a8db1724eaba0187e06f77a14f43a7f1dc7db60d8828ad9b066cf99532d: Status 404 returned error can't find the container with id da1c1a8db1724eaba0187e06f77a14f43a7f1dc7db60d8828ad9b066cf99532d Jun 08 18:15:27.329052 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.329037 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 18:15:27.334747 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.334708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" event={"ID":"b0bdb72680f38d3f1fa35164d90fd6e0","Type":"ContainerStarted","Data":"da1c1a8db1724eaba0187e06f77a14f43a7f1dc7db60d8828ad9b066cf99532d"} Jun 08 18:15:27.336258 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.336238 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" event={"ID":"05b840dcb7cfc31951c9194a12f1eb48","Type":"ContainerStarted","Data":"b4303c4e099e0487b4eaee2c32acb2c94e0b50167106bcc24917eff9a506032b"} Jun 08 18:15:27.371155 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:27.371117 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:27.471625 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:27.471584 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:27.572116 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:27.572087 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:27.672649 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:27.672580 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jun 08 18:15:27.736282 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.736107 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:15:27.762104 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.762074 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jun 08 18:15:27.773409 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.773382 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 18:15:27.774911 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.774722 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jun 08 18:15:27.783400 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:27.783379 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 18:15:28.008470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.008399 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:15:28.234285 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.234236 2560 apiserver.go:52] "Watching apiserver" Jun 08 18:15:28.242217 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.242195 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 08 18:15:28.243438 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.243413 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-nk59b","openshift-network-diagnostics/network-check-target-p99xb","openshift-network-operator/iptables-alerter-wz62g","kube-system/global-pull-secret-syncer-lbhcf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg","openshift-dns/node-resolver-mqbnh","openshift-image-registry/node-ca-8cs4k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-b4xn9","kube-system/konnectivity-agent-vl77q","kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal","openshift-cluster-node-tuning-operator/tuned-dkzs2","openshift-multus/multus-additional-cni-plugins-26l7d","openshift-multus/multus-vslwg"] Jun 08 18:15:28.245045 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.245028 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.246294 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.246269 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.247504 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.247484 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 08 18:15:28.247604 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.247546 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 08 18:15:28.247655 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.247609 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.247655 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.247484 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 08 18:15:28.247979 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.247963 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6bp85\"" Jun 08 18:15:28.248430 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.248412 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 08 18:15:28.248505 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.248430 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 08 18:15:28.248560 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.248515 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 08 18:15:28.248610 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.248556 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k4xll\"" Jun 08 18:15:28.248701 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.248670 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 08 18:15:28.248825 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.248766 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.248895 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.248834 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:28.248954 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.248927 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 08 18:15:28.249739 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.249718 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 08 18:15:28.249739 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.249731 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 08 18:15:28.249950 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.249870 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jkvw4\"" Jun 08 18:15:28.250074 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.250050 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:15:28.250177 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.250138 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.251351 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.251332 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.252619 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.252601 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 08 18:15:28.252711 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.252683 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 08 18:15:28.252967 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.252951 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.254485 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.254245 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 08 18:15:28.254485 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.254462 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-45f4g\"" Jun 08 18:15:28.254682 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.254546 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 08 18:15:28.254682 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.254600 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 08 18:15:28.254790 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.254690 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dqfhl\"" Jun 08 18:15:28.254888 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.254869 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:28.254979 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.254930 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:28.255108 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.255092 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 08 18:15:28.255372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.255359 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 08 18:15:28.255372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.255366 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6jk7d\"" Jun 08 18:15:28.256403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.256383 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 08 18:15:28.256704 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.256687 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 08 18:15:28.257455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.257174 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 08 18:15:28.257542 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.257512 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 08 18:15:28.257769 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.257750 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:28.257881 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.257858 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:28.259620 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.259222 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:28.260823 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.260806 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.261757 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.261738 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 08 18:15:28.262015 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.261960 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 08 18:15:28.262164 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.262149 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-95bhp\"" Jun 08 18:15:28.262389 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.262367 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.262822 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.262806 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 08 18:15:28.263050 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.263030 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 08 18:15:28.263299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.263279 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fns5s\"" Jun 08 18:15:28.263926 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.263908 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 08 18:15:28.264500 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.264481 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:15:28.264905 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.264887 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 08 18:15:28.265059 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.265041 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fs6pp\"" Jun 08 18:15:28.272337 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272318 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-socket-dir-parent\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.272442 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272348 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-conf-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.272442 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272371 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.272442 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-registration-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.272442 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272431 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysconfig\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.272651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272484 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-cni-netd\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.272651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272522 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovnkube-script-lib\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.272651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272549 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-system-cni-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.272651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-hostroot\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.272651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272607 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-etc-kubernetes\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.272651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-device-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272676 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272698 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-slash\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272721 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-ovn\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-cni-bin\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k4l7\" (UniqueName: \"kubernetes.io/projected/9dd12107-4224-4236-87ab-2d5f533e8c4d-kube-api-access-5k4l7\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272790 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-log-socket\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-cni-bin\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-systemd\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272897 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-kubelet\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpp8z\" (UniqueName: \"kubernetes.io/projected/6da4b355-be75-4cfb-93f2-a6289976c79b-kube-api-access-tpp8z\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.272941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272937 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cnibin\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272961 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cni-binary-copy\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.272980 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-cni-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273001 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02733b0c-c0c5-422b-8d01-de38f405ef33-hosts-file\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273023 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xp7t\" (UniqueName: \"kubernetes.io/projected/e5c81386-a373-411b-9eb1-205634bdae58-kube-api-access-2xp7t\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273054 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-kubernetes\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273104 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-env-overrides\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovn-node-metrics-cert\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273189 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9dd12107-4224-4236-87ab-2d5f533e8c4d-iptables-alerter-script\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273235 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796c82e4-e1bb-436b-9145-8377aed43b06-dbus\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.273340 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-node-log\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-cnibin\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273392 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-tuned\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-host\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796c82e4-e1bb-436b-9145-8377aed43b06-kubelet-config\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273522 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa-agent-certs\") pod \"konnectivity-agent-vl77q\" (UID: \"ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa\") " pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273547 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysctl-conf\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273569 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-os-release\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273591 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b88dk\" (UniqueName: \"kubernetes.io/projected/d1540b0a-c591-4239-8ac2-41fd39c3eae3-kube-api-access-b88dk\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273636 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.273694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273672 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-os-release\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273704 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-host\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273751 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lprv7\" (UniqueName: \"kubernetes.io/projected/3985b2b9-e6b3-4430-ae83-217a9afde35f-kube-api-access-lprv7\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-kubelet\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273800 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-systemd-units\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-etc-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-socket-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-sys-fs\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273926 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273956 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovnkube-config\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273979 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2w6p\" (UniqueName: \"kubernetes.io/projected/f4232b10-00c4-4f03-b7fa-d602b432a60c-kube-api-access-s2w6p\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.273995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkw22\" (UniqueName: \"kubernetes.io/projected/02733b0c-c0c5-422b-8d01-de38f405ef33-kube-api-access-dkw22\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274009 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-modprobe-d\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-sys\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.274219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274059 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-lib-modules\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jtxp\" (UniqueName: \"kubernetes.io/projected/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-kube-api-access-6jtxp\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa-konnectivity-ca\") pod \"konnectivity-agent-vl77q\" (UID: \"ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa\") " pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274248 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4232b10-00c4-4f03-b7fa-d602b432a60c-cni-binary-copy\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274286 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-daemon-config\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274318 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-systemd\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274342 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-system-cni-dir\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-var-lib-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274424 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-run\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274450 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-tmp\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274472 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9dd12107-4224-4236-87ab-2d5f533e8c4d-host-slash\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274531 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-k8s-cni-cncf-io\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274553 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-netns\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274575 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-cni-multus\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-multus-certs\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.274893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274621 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysctl-d\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.275415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvznt\" (UniqueName: \"kubernetes.io/projected/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-kube-api-access-pvznt\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.275415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274680 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02733b0c-c0c5-422b-8d01-de38f405ef33-tmp-dir\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.275415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-var-lib-kubelet\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.275415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-serviceca\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.275415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274736 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.275415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.274765 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-run-netns\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.275415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.275292 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 18:15:28.293749 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.293729 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xlcpg" Jun 08 18:15:28.301428 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.301410 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xlcpg" Jun 08 18:15:28.375939 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.375904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-socket-dir-parent\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376094 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.375950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-conf-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376094 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.375974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.376094 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.375989 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-registration-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.376094 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysconfig\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.376094 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376011 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-socket-dir-parent\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376094 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376076 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.376094 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376075 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-conf-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376113 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysconfig\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376135 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-cni-netd\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376151 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-registration-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovnkube-script-lib\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376187 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-cni-netd\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-system-cni-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-hostroot\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376239 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-etc-kubernetes\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376262 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-device-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376291 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376303 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-hostroot\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-slash\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376333 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-system-cni-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376334 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-etc-kubernetes\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376357 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-slash\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-ovn\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.376372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376376 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-ovn\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376389 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-cni-bin\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376399 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376401 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-device-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k4l7\" (UniqueName: \"kubernetes.io/projected/9dd12107-4224-4236-87ab-2d5f533e8c4d-kube-api-access-5k4l7\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376428 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-cni-bin\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376466 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-log-socket\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-cni-bin\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-systemd\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376567 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-log-socket\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-kubelet\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376587 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-cni-bin\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376608 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-run-systemd\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpp8z\" (UniqueName: \"kubernetes.io/projected/6da4b355-be75-4cfb-93f2-a6289976c79b-kube-api-access-tpp8z\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376630 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-kubelet\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cnibin\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cni-binary-copy\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.377223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovnkube-script-lib\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-cni-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376718 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cnibin\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02733b0c-c0c5-422b-8d01-de38f405ef33-hosts-file\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xp7t\" (UniqueName: \"kubernetes.io/projected/e5c81386-a373-411b-9eb1-205634bdae58-kube-api-access-2xp7t\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376765 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-cni-dir\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376770 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-kubernetes\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376800 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-env-overrides\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376817 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-kubernetes\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376824 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovn-node-metrics-cert\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376837 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02733b0c-c0c5-422b-8d01-de38f405ef33-hosts-file\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9dd12107-4224-4236-87ab-2d5f533e8c4d-iptables-alerter-script\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796c82e4-e1bb-436b-9145-8377aed43b06-dbus\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-node-log\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376913 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-cnibin\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376946 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-tuned\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376977 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-host\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.378031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.376991 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377006 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796c82e4-e1bb-436b-9145-8377aed43b06-kubelet-config\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa-agent-certs\") pod \"konnectivity-agent-vl77q\" (UID: \"ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa\") " pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377042 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysctl-conf\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377050 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-cnibin\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-os-release\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377078 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b88dk\" (UniqueName: \"kubernetes.io/projected/d1540b0a-c591-4239-8ac2-41fd39c3eae3-kube-api-access-b88dk\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377118 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377144 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-os-release\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377154 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377174 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796c82e4-e1bb-436b-9145-8377aed43b06-kubelet-config\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377204 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-host\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377262 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cni-binary-copy\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lprv7\" (UniqueName: \"kubernetes.io/projected/3985b2b9-e6b3-4430-ae83-217a9afde35f-kube-api-access-lprv7\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377277 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-env-overrides\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:28.378918 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.377309 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377322 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796c82e4-e1bb-436b-9145-8377aed43b06-dbus\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377331 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-kubelet\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377334 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-systemd-units\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377378 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-kubelet\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.377398 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret podName:796c82e4-e1bb-436b-9145-8377aed43b06 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:28.87735925 +0000 UTC m=+3.038195903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret") pod "global-pull-secret-syncer-lbhcf" (UID: "796c82e4-e1bb-436b-9145-8377aed43b06") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377401 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-systemd-units\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-etc-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377431 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-os-release\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.377468 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377473 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-etc-selinux\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377532 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9dd12107-4224-4236-87ab-2d5f533e8c4d-iptables-alerter-script\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377588 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-etc-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377617 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-socket-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377620 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysctl-conf\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377629 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-os-release\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.379716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-sys-fs\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377656 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-host\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377663 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377667 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-node-log\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377691 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovnkube-config\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377716 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-host\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377743 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-sys-fs\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.377786 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:28.877768536 +0000 UTC m=+3.038605190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377785 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377796 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6da4b355-be75-4cfb-93f2-a6289976c79b-socket-dir\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2w6p\" (UniqueName: \"kubernetes.io/projected/f4232b10-00c4-4f03-b7fa-d602b432a60c-kube-api-access-s2w6p\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.377897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkw22\" (UniqueName: \"kubernetes.io/projected/02733b0c-c0c5-422b-8d01-de38f405ef33-kube-api-access-dkw22\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-modprobe-d\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378098 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-sys\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378120 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-lib-modules\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jtxp\" (UniqueName: \"kubernetes.io/projected/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-kube-api-access-6jtxp\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378167 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa-konnectivity-ca\") pod \"konnectivity-agent-vl77q\" (UID: \"ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa\") " pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:28.380590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378197 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4232b10-00c4-4f03-b7fa-d602b432a60c-cni-binary-copy\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-daemon-config\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378229 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-sys\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378237 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378268 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovnkube-config\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378297 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-systemd\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378367 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-modprobe-d\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378378 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-lib-modules\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-systemd\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378433 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-system-cni-dir\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378468 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-var-lib-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-run\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-tmp\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378561 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-var-lib-openvswitch\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378607 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9dd12107-4224-4236-87ab-2d5f533e8c4d-host-slash\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378610 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-run\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378657 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-system-cni-dir\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9dd12107-4224-4236-87ab-2d5f533e8c4d-host-slash\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378694 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-k8s-cni-cncf-io\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-netns\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378740 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-cni-multus\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-multus-certs\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378781 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysctl-d\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvznt\" (UniqueName: \"kubernetes.io/projected/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-kube-api-access-pvznt\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378821 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02733b0c-c0c5-422b-8d01-de38f405ef33-tmp-dir\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378834 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa-konnectivity-ca\") pod \"konnectivity-agent-vl77q\" (UID: \"ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa\") " pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-var-lib-kubelet\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378879 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4232b10-00c4-4f03-b7fa-d602b432a60c-cni-binary-copy\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378887 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-serviceca\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378940 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4232b10-00c4-4f03-b7fa-d602b432a60c-multus-daemon-config\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-run-netns\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.378993 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1540b0a-c591-4239-8ac2-41fd39c3eae3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379266 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-sysctl-d\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.381956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379283 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02733b0c-c0c5-422b-8d01-de38f405ef33-tmp-dir\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379307 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-serviceca\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379334 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-var-lib-kubelet\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379365 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-k8s-cni-cncf-io\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379410 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1540b0a-c591-4239-8ac2-41fd39c3eae3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379408 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-netns\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379443 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-run-multus-certs\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379458 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3985b2b9-e6b3-4430-ae83-217a9afde35f-host-run-netns\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.379459 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4232b10-00c4-4f03-b7fa-d602b432a60c-host-var-lib-cni-multus\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.380883 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-etc-tuned\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.381039 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3985b2b9-e6b3-4430-ae83-217a9afde35f-ovn-node-metrics-cert\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.381361 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-tmp\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.382676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.381362 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa-agent-certs\") pod \"konnectivity-agent-vl77q\" (UID: \"ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa\") " pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:28.387600 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.387509 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpp8z\" (UniqueName: \"kubernetes.io/projected/6da4b355-be75-4cfb-93f2-a6289976c79b-kube-api-access-tpp8z\") pod \"aws-ebs-csi-driver-node-hncdg\" (UID: \"6da4b355-be75-4cfb-93f2-a6289976c79b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.387600 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.387509 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b88dk\" (UniqueName: \"kubernetes.io/projected/d1540b0a-c591-4239-8ac2-41fd39c3eae3-kube-api-access-b88dk\") pod \"multus-additional-cni-plugins-26l7d\" (UID: \"d1540b0a-c591-4239-8ac2-41fd39c3eae3\") " pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.387779 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.387606 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:15:28.387779 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.387629 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:15:28.387779 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.387643 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kr5vn for pod openshift-network-diagnostics/network-check-target-p99xb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:28.387779 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.387696 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn podName:9ed73b24-9670-44b6-84f2-9a645841edf2 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:28.887679418 +0000 UTC m=+3.048516055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kr5vn" (UniqueName: "kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn") pod "network-check-target-p99xb" (UID: "9ed73b24-9670-44b6-84f2-9a645841edf2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:28.387779 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.387609 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xp7t\" (UniqueName: \"kubernetes.io/projected/e5c81386-a373-411b-9eb1-205634bdae58-kube-api-access-2xp7t\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:28.390680 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.390630 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jtxp\" (UniqueName: \"kubernetes.io/projected/c4835af7-f4d0-4820-ba05-41ac0bcf4b66-kube-api-access-6jtxp\") pod \"node-ca-8cs4k\" (UID: \"c4835af7-f4d0-4820-ba05-41ac0bcf4b66\") " pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.390680 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.390649 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2w6p\" (UniqueName: \"kubernetes.io/projected/f4232b10-00c4-4f03-b7fa-d602b432a60c-kube-api-access-s2w6p\") pod \"multus-vslwg\" (UID: \"f4232b10-00c4-4f03-b7fa-d602b432a60c\") " pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.390825 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.390701 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lprv7\" (UniqueName: \"kubernetes.io/projected/3985b2b9-e6b3-4430-ae83-217a9afde35f-kube-api-access-lprv7\") pod \"ovnkube-node-b4xn9\" (UID: \"3985b2b9-e6b3-4430-ae83-217a9afde35f\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.390825 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.390775 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvznt\" (UniqueName: \"kubernetes.io/projected/42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1-kube-api-access-pvznt\") pod \"tuned-dkzs2\" (UID: \"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1\") " pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.391220 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.391192 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k4l7\" (UniqueName: \"kubernetes.io/projected/9dd12107-4224-4236-87ab-2d5f533e8c4d-kube-api-access-5k4l7\") pod \"iptables-alerter-wz62g\" (UID: \"9dd12107-4224-4236-87ab-2d5f533e8c4d\") " pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.392447 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.392424 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkw22\" (UniqueName: \"kubernetes.io/projected/02733b0c-c0c5-422b-8d01-de38f405ef33-kube-api-access-dkw22\") pod \"node-resolver-mqbnh\" (UID: \"02733b0c-c0c5-422b-8d01-de38f405ef33\") " pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.558051 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.557944 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8cs4k" Jun 08 18:15:28.565815 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.565788 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-26l7d" Jun 08 18:15:28.576155 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.576122 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wz62g" Jun 08 18:15:28.580307 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.580289 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" Jun 08 18:15:28.587209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.587191 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mqbnh" Jun 08 18:15:28.594970 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.594953 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:28.601889 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.601872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:28.608892 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.608872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vslwg" Jun 08 18:15:28.613748 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.613732 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" Jun 08 18:15:28.623000 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.622981 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:15:28.883302 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.883221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:28.883302 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.883275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:28.883525 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.883390 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:28.883525 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.883421 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:28.883525 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.883470 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:29.883450323 +0000 UTC m=+4.044286974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:28.883525 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.883489 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret podName:796c82e4-e1bb-436b-9145-8377aed43b06 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:29.883479829 +0000 UTC m=+4.044316464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret") pod "global-pull-secret-syncer-lbhcf" (UID: "796c82e4-e1bb-436b-9145-8377aed43b06") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:28.984577 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:28.984546 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:28.984743 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.984710 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:15:28.984743 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.984730 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:15:28.984743 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.984740 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kr5vn for pod openshift-network-diagnostics/network-check-target-p99xb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:28.984864 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:28.984792 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn podName:9ed73b24-9670-44b6-84f2-9a645841edf2 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:29.984778404 +0000 UTC m=+4.145615040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kr5vn" (UniqueName: "kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn") pod "network-check-target-p99xb" (UID: "9ed73b24-9670-44b6-84f2-9a645841edf2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:29.141723 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:29.141686 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02733b0c_c0c5_422b_8d01_de38f405ef33.slice/crio-1646540d9578cc41d4b17441b27596609f18a10e1b6ffc4f278d013bd8e18ea1 WatchSource:0}: Error finding container 1646540d9578cc41d4b17441b27596609f18a10e1b6ffc4f278d013bd8e18ea1: Status 404 returned error can't find the container with id 1646540d9578cc41d4b17441b27596609f18a10e1b6ffc4f278d013bd8e18ea1 Jun 08 18:15:29.142920 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:29.142855 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3985b2b9_e6b3_4430_ae83_217a9afde35f.slice/crio-e00d6d6860a381f93c01cb8352267c8a99466ec17f7840a37c2dbd9788e1a05f WatchSource:0}: Error finding container e00d6d6860a381f93c01cb8352267c8a99466ec17f7840a37c2dbd9788e1a05f: Status 404 returned error can't find the container with id e00d6d6860a381f93c01cb8352267c8a99466ec17f7840a37c2dbd9788e1a05f Jun 08 18:15:29.149460 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:29.149431 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1540b0a_c591_4239_8ac2_41fd39c3eae3.slice/crio-bc77f9b9ea4964fe6a5f603a8564ebbce69dd581de9298a60383bf94fa9bd8b4 WatchSource:0}: Error finding container bc77f9b9ea4964fe6a5f603a8564ebbce69dd581de9298a60383bf94fa9bd8b4: Status 404 returned error can't find the container with id bc77f9b9ea4964fe6a5f603a8564ebbce69dd581de9298a60383bf94fa9bd8b4 Jun 08 18:15:29.149884 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:29.149860 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0ccf9f_a60b_4b52_8e2a_e4a420c337fa.slice/crio-392cba76d4c092b5328ad3af01dab3237cddf5af654cbaf439b932df57d4a505 WatchSource:0}: Error finding container 392cba76d4c092b5328ad3af01dab3237cddf5af654cbaf439b932df57d4a505: Status 404 returned error can't find the container with id 392cba76d4c092b5328ad3af01dab3237cddf5af654cbaf439b932df57d4a505 Jun 08 18:15:29.151524 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:29.151294 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4835af7_f4d0_4820_ba05_41ac0bcf4b66.slice/crio-39a29abc347a0146d95fc11c6eaf827f868914f3c68e91ed60af6880cb09bdf3 WatchSource:0}: Error finding container 39a29abc347a0146d95fc11c6eaf827f868914f3c68e91ed60af6880cb09bdf3: Status 404 returned error can't find the container with id 39a29abc347a0146d95fc11c6eaf827f868914f3c68e91ed60af6880cb09bdf3 Jun 08 18:15:29.153367 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:15:29.153347 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da4b355_be75_4cfb_93f2_a6289976c79b.slice/crio-3d9f847cf30f30af104d4336b14eb0c75628a7e71ccd261b7edce3b7474f5229 WatchSource:0}: Error finding container 3d9f847cf30f30af104d4336b14eb0c75628a7e71ccd261b7edce3b7474f5229: Status 404 returned error can't find the container with id 3d9f847cf30f30af104d4336b14eb0c75628a7e71ccd261b7edce3b7474f5229 Jun 08 18:15:29.303279 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.303232 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 18:10:28 +0000 UTC" deadline="2027-11-27 04:24:07.422896858 +0000 UTC" Jun 08 18:15:29.303279 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.303274 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12874h8m38.11962524s" Jun 08 18:15:29.331685 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.331664 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:29.331806 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.331752 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:29.339732 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.339707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vslwg" event={"ID":"f4232b10-00c4-4f03-b7fa-d602b432a60c","Type":"ContainerStarted","Data":"e2a88ad8cfbb565ca65e3675beb4fa73a1d89871201a8d5ce7256be59df7ca8f"} Jun 08 18:15:29.340892 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.340869 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" event={"ID":"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1","Type":"ContainerStarted","Data":"08333b0be0e06105b249d0bbebfef74e38f22010ce7ca25509536a02081de843"} Jun 08 18:15:29.341890 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.341867 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8cs4k" event={"ID":"c4835af7-f4d0-4820-ba05-41ac0bcf4b66","Type":"ContainerStarted","Data":"39a29abc347a0146d95fc11c6eaf827f868914f3c68e91ed60af6880cb09bdf3"} Jun 08 18:15:29.342980 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.342956 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vl77q" event={"ID":"ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa","Type":"ContainerStarted","Data":"392cba76d4c092b5328ad3af01dab3237cddf5af654cbaf439b932df57d4a505"} Jun 08 18:15:29.343952 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.343931 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"e00d6d6860a381f93c01cb8352267c8a99466ec17f7840a37c2dbd9788e1a05f"} Jun 08 18:15:29.345803 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.345782 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" event={"ID":"b0bdb72680f38d3f1fa35164d90fd6e0","Type":"ContainerStarted","Data":"dcd0b5ae12e4690cbe9a169b04a172e959150ae6ca6c9e37f9b440b056da525e"} Jun 08 18:15:29.346829 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.346811 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" event={"ID":"6da4b355-be75-4cfb-93f2-a6289976c79b","Type":"ContainerStarted","Data":"3d9f847cf30f30af104d4336b14eb0c75628a7e71ccd261b7edce3b7474f5229"} Jun 08 18:15:29.350103 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.350086 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wz62g" event={"ID":"9dd12107-4224-4236-87ab-2d5f533e8c4d","Type":"ContainerStarted","Data":"bfac840b6a7e2743ad8374f6130eb420c34ea22af1362eb4d58bd1d062fe64be"} Jun 08 18:15:29.351815 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.351787 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26l7d" event={"ID":"d1540b0a-c591-4239-8ac2-41fd39c3eae3","Type":"ContainerStarted","Data":"bc77f9b9ea4964fe6a5f603a8564ebbce69dd581de9298a60383bf94fa9bd8b4"} Jun 08 18:15:29.352773 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.352755 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mqbnh" event={"ID":"02733b0c-c0c5-422b-8d01-de38f405ef33","Type":"ContainerStarted","Data":"1646540d9578cc41d4b17441b27596609f18a10e1b6ffc4f278d013bd8e18ea1"} Jun 08 18:15:29.363319 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.363280 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" podStartSLOduration=2.363265988 podStartE2EDuration="2.363265988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:29.363077868 +0000 UTC m=+3.523914526" watchObservedRunningTime="2026-06-08 18:15:29.363265988 +0000 UTC m=+3.524102646" Jun 08 18:15:29.640966 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.640917 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:15:29.893525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.893484 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:29.893689 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.893545 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:29.893744 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.893694 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:29.893792 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.893755 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret podName:796c82e4-e1bb-436b-9145-8377aed43b06 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:31.893737801 +0000 UTC m=+6.054574442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret") pod "global-pull-secret-syncer-lbhcf" (UID: "796c82e4-e1bb-436b-9145-8377aed43b06") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:29.893888 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.893817 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:29.893888 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.893870 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:31.893837944 +0000 UTC m=+6.054674585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:29.994659 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:29.994618 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:29.994835 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.994789 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:15:29.994835 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.994812 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:15:29.994835 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.994825 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kr5vn for pod openshift-network-diagnostics/network-check-target-p99xb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:29.995039 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:29.994897 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn podName:9ed73b24-9670-44b6-84f2-9a645841edf2 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:31.994878577 +0000 UTC m=+6.155715220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kr5vn" (UniqueName: "kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn") pod "network-check-target-p99xb" (UID: "9ed73b24-9670-44b6-84f2-9a645841edf2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:30.303654 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:30.303564 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 18:10:28 +0000 UTC" deadline="2028-02-19 12:20:49.528144134 +0000 UTC" Jun 08 18:15:30.303654 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:30.303605 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14898h5m19.224543648s" Jun 08 18:15:30.334008 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:30.333974 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:30.334008 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:30.333982 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:30.334212 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:30.334118 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:30.334352 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:30.334328 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:30.365548 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:30.365512 2560 generic.go:356] "Generic (PLEG): container finished" podID="05b840dcb7cfc31951c9194a12f1eb48" containerID="debbaf76a6a3c54c51ba162c04795bcd8f4858990be9c9935eb950eafaefd6bb" exitCode=0 Jun 08 18:15:30.365703 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:30.365613 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" event={"ID":"05b840dcb7cfc31951c9194a12f1eb48","Type":"ContainerDied","Data":"debbaf76a6a3c54c51ba162c04795bcd8f4858990be9c9935eb950eafaefd6bb"} Jun 08 18:15:31.331859 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:31.331811 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:31.332318 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:31.331955 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:31.377872 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:31.377142 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" event={"ID":"05b840dcb7cfc31951c9194a12f1eb48","Type":"ContainerStarted","Data":"245c76d119f8abff09413ee87f5f2940f855e0efd4c2ee2b65dd54493108bb47"} Jun 08 18:15:31.394358 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:31.393574 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" podStartSLOduration=4.393557917 podStartE2EDuration="4.393557917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:31.393428523 +0000 UTC m=+5.554265181" watchObservedRunningTime="2026-06-08 18:15:31.393557917 +0000 UTC m=+5.554394578" Jun 08 18:15:31.910655 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:31.910620 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:31.910813 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:31.910674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:31.910813 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:31.910804 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:31.910956 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:31.910884 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret podName:796c82e4-e1bb-436b-9145-8377aed43b06 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:35.910864852 +0000 UTC m=+10.071701502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret") pod "global-pull-secret-syncer-lbhcf" (UID: "796c82e4-e1bb-436b-9145-8377aed43b06") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:31.911285 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:31.911263 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:31.911363 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:31.911326 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:35.911308752 +0000 UTC m=+10.072145415 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:32.012042 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:32.011434 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:32.012042 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:32.011610 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:15:32.012042 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:32.011629 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:15:32.012042 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:32.011638 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kr5vn for pod openshift-network-diagnostics/network-check-target-p99xb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:32.012042 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:32.011678 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn podName:9ed73b24-9670-44b6-84f2-9a645841edf2 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:36.011666577 +0000 UTC m=+10.172503225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kr5vn" (UniqueName: "kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn") pod "network-check-target-p99xb" (UID: "9ed73b24-9670-44b6-84f2-9a645841edf2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:32.332189 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:32.332153 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:32.332610 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:32.332289 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:32.332698 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:32.332677 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:32.332801 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:32.332777 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:33.332879 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:33.332462 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:33.332879 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:33.332594 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:34.335640 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:34.335608 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:34.336027 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:34.335726 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:34.336027 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:34.335923 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:34.336027 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:34.336014 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:35.332186 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:35.332147 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:35.332354 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:35.332277 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:35.943496 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:35.943457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:35.943910 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:35.943568 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:35.943910 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:35.943728 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:35.943910 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:35.943787 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:43.943768139 +0000 UTC m=+18.104604792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:35.947925 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:35.947900 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:35.948059 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:35.947977 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret podName:796c82e4-e1bb-436b-9145-8377aed43b06 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:43.947957547 +0000 UTC m=+18.108794189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret") pod "global-pull-secret-syncer-lbhcf" (UID: "796c82e4-e1bb-436b-9145-8377aed43b06") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:36.044868 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:36.044322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:36.044868 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:36.044503 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:15:36.044868 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:36.044530 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:15:36.044868 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:36.044543 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kr5vn for pod openshift-network-diagnostics/network-check-target-p99xb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:36.044868 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:36.044588 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn podName:9ed73b24-9670-44b6-84f2-9a645841edf2 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:44.044574112 +0000 UTC m=+18.205410748 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kr5vn" (UniqueName: "kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn") pod "network-check-target-p99xb" (UID: "9ed73b24-9670-44b6-84f2-9a645841edf2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:36.333786 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:36.333194 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:36.333786 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:36.333328 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:36.334059 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:36.333925 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:36.334998 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:36.334965 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:37.332263 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:37.332223 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:37.332719 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:37.332355 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:38.332198 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:38.332161 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:38.332449 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:38.332302 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:38.332796 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:38.332719 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:38.332867 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:38.332832 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:39.331912 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:39.331876 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:39.332082 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:39.332013 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:40.332469 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:40.332260 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:40.332949 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:40.332584 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:40.333027 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:40.332323 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:40.333078 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:40.333059 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:41.331645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:41.331613 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:41.331783 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:41.331721 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:42.332459 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:42.332427 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:42.332881 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:42.332442 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:42.332881 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:42.332572 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:42.332881 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:42.332601 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:43.332058 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:43.332027 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:43.332214 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:43.332150 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:44.004950 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:44.004829 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:44.005403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:44.005015 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:44.005403 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.005356 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:44.005576 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.005561 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret podName:796c82e4-e1bb-436b-9145-8377aed43b06 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:00.005544964 +0000 UTC m=+34.166381604 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret") pod "global-pull-secret-syncer-lbhcf" (UID: "796c82e4-e1bb-436b-9145-8377aed43b06") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:15:44.007409 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.006946 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:44.007409 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.007126 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:00.00709796 +0000 UTC m=+34.167934608 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:15:44.105973 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:44.105941 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:44.106130 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.106112 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:15:44.106211 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.106141 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:15:44.106211 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.106155 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kr5vn for pod openshift-network-diagnostics/network-check-target-p99xb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:44.106316 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.106233 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn podName:9ed73b24-9670-44b6-84f2-9a645841edf2 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:00.106213911 +0000 UTC m=+34.267050550 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kr5vn" (UniqueName: "kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn") pod "network-check-target-p99xb" (UID: "9ed73b24-9670-44b6-84f2-9a645841edf2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:15:44.332341 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:44.332310 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:44.332505 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:44.332311 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:44.332505 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.332433 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:44.332614 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:44.332529 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:45.332207 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:45.332175 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:45.332684 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:45.332304 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:46.334515 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:46.334487 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:46.334968 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:46.334608 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:46.334968 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:46.334620 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:46.334968 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:46.334729 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:47.332211 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:47.331740 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:47.332211 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:47.331879 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:48.331837 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.331634 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:48.332601 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.331635 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:48.332601 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:48.331948 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:48.332601 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:48.332050 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:48.409732 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.409694 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" event={"ID":"6da4b355-be75-4cfb-93f2-a6289976c79b","Type":"ContainerStarted","Data":"242ad8fb0fb839365221e7def542e0d9b9c2c2e21a971fa13daac2686a71ed16"} Jun 08 18:15:48.411583 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.411561 2560 generic.go:356] "Generic (PLEG): container finished" podID="d1540b0a-c591-4239-8ac2-41fd39c3eae3" containerID="8bb3c924ad203bc554c294c940b7869f380905e987121e06fbeabc472063a0e6" exitCode=0 Jun 08 18:15:48.411697 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.411642 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26l7d" event={"ID":"d1540b0a-c591-4239-8ac2-41fd39c3eae3","Type":"ContainerDied","Data":"8bb3c924ad203bc554c294c940b7869f380905e987121e06fbeabc472063a0e6"} Jun 08 18:15:48.415065 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.415002 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mqbnh" event={"ID":"02733b0c-c0c5-422b-8d01-de38f405ef33","Type":"ContainerStarted","Data":"c1ae088a9f35a6dc625b18a003d498a80747ce71501c714908cb2b75f780deae"} Jun 08 18:15:48.416374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.416350 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vslwg" event={"ID":"f4232b10-00c4-4f03-b7fa-d602b432a60c","Type":"ContainerStarted","Data":"984ef9c17b9d74b6f79a57fce9fe6c524af8531b3468d23a25cd6a1f8f2139c2"} Jun 08 18:15:48.418211 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.418192 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" event={"ID":"42ff21b5-7c8e-4cc6-8e2a-145192d7a9a1","Type":"ContainerStarted","Data":"616fe1bde65e4d6beabaef0167f937274da225e6dd155cd876c8b007d591c2a7"} Jun 08 18:15:48.419672 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.419650 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8cs4k" event={"ID":"c4835af7-f4d0-4820-ba05-41ac0bcf4b66","Type":"ContainerStarted","Data":"799f1705c3c83f8f6da5306b20179de06ea7940b74d0b97d55aa0dbb1982b616"} Jun 08 18:15:48.421225 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.421200 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vl77q" event={"ID":"ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa","Type":"ContainerStarted","Data":"9844fe437cb6a78b823f680c74818c78a125bd327955cf60bb82ecba98b85f62"} Jun 08 18:15:48.423898 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.423878 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:15:48.424313 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.424285 2560 generic.go:356] "Generic (PLEG): container finished" podID="3985b2b9-e6b3-4430-ae83-217a9afde35f" containerID="247173a5aafc51241fbb6d1965ad0de38133fc5669c5c5adb9424a9233f47998" exitCode=1 Jun 08 18:15:48.424396 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.424327 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"63e02e8523af546c34d0275adcdfebb84ba15ee00479c85acdb0f92ad3c15866"} Jun 08 18:15:48.424448 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.424440 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerDied","Data":"247173a5aafc51241fbb6d1965ad0de38133fc5669c5c5adb9424a9233f47998"} Jun 08 18:15:48.424522 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.424504 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"76d36c8ae4f2d7f5122737fdd3dbf24cf851e87e27f99531e0ef58aeb649371f"} Jun 08 18:15:48.471959 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.471922 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vl77q" podStartSLOduration=8.254215019 podStartE2EDuration="22.47190879s" totalImagesPullingTime="14.217693771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:48.471772748 +0000 UTC m=+22.632609408" watchObservedRunningTime="2026-06-08 18:15:48.47190879 +0000 UTC m=+22.632745448" Jun 08 18:15:48.472385 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.472321 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mqbnh" podStartSLOduration=3.682121785 podStartE2EDuration="22.472313551s" totalImagesPullingTime="18.790191766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:48.455998818 +0000 UTC m=+22.616835486" watchObservedRunningTime="2026-06-08 18:15:48.472313551 +0000 UTC m=+22.633150209" Jun 08 18:15:48.485893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.485833 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8cs4k" podStartSLOduration=3.731993783 podStartE2EDuration="22.48581924s" totalImagesPullingTime="18.753825457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:48.48560329 +0000 UTC m=+22.646439949" watchObservedRunningTime="2026-06-08 18:15:48.48581924 +0000 UTC m=+22.646655897" Jun 08 18:15:48.517902 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.517837 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dkzs2" podStartSLOduration=3.738959085 podStartE2EDuration="22.517821251s" totalImagesPullingTime="18.778862166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:48.517525516 +0000 UTC m=+22.678362174" watchObservedRunningTime="2026-06-08 18:15:48.517821251 +0000 UTC m=+22.678657930" Jun 08 18:15:48.518075 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:48.518048 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vslwg" podStartSLOduration=3.718710229 podStartE2EDuration="22.518042254s" totalImagesPullingTime="18.799332025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:48.500490963 +0000 UTC m=+22.661327621" watchObservedRunningTime="2026-06-08 18:15:48.518042254 +0000 UTC m=+22.678878914" Jun 08 18:15:49.244970 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.244742 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 08 18:15:49.331854 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.331829 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:49.331977 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:49.331939 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:49.339857 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.339769 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-08T18:15:49.244967118Z","UUID":"137e890c-8a84-44cc-8155-29f70d80b595","Handler":null,"Name":"","Endpoint":""} Jun 08 18:15:49.341115 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.341099 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 08 18:15:49.341206 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.341135 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 08 18:15:49.429961 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.429939 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:15:49.430514 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.430311 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"9742d103fedf6f25c873b72d3229f9dbdce7f35cdc10e82eedbccbf8dabb6c94"} Jun 08 18:15:49.430514 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.430393 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"002aab64456287f99101d13ff48a9c137d8d0e95302388d0c2c39c832e3e68b6"} Jun 08 18:15:49.430514 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.430443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"d83c4815aaf1e9a758c5e6c4f9bd94b7960a4ab05b508e2e2134cc89e51eea95"} Jun 08 18:15:49.432822 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.432795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" event={"ID":"6da4b355-be75-4cfb-93f2-a6289976c79b","Type":"ContainerStarted","Data":"c40a158f2611eeb406d99950a8768b7713694aac49276f4ed5e72ed1e3d27e92"} Jun 08 18:15:49.436380 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.436356 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wz62g" event={"ID":"9dd12107-4224-4236-87ab-2d5f533e8c4d","Type":"ContainerStarted","Data":"9290c87d48191e677c9a018c74c6956f1dd622c34140de211e5732e3518a9b5b"} Jun 08 18:15:49.458906 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.458870 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wz62g" podStartSLOduration=4.678990204 podStartE2EDuration="23.458835753s" totalImagesPullingTime="18.779845549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:49.458579356 +0000 UTC m=+23.619416015" watchObservedRunningTime="2026-06-08 18:15:49.458835753 +0000 UTC m=+23.619672410" Jun 08 18:15:49.677196 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:49.677110 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:50.332572 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:50.332533 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:50.333170 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:50.332670 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:50.333170 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:50.332753 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:50.333170 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:50.332885 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:50.442826 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:50.442788 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" event={"ID":"6da4b355-be75-4cfb-93f2-a6289976c79b","Type":"ContainerStarted","Data":"d04d535ca918793fb44be348752f6651d00adb0f09d9a295420ad8929ebfa95a"} Jun 08 18:15:50.464754 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:50.464708 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hncdg" podStartSLOduration=4.016126964 podStartE2EDuration="24.464695052s" totalImagesPullingTime="20.448568088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:50.46441046 +0000 UTC m=+24.625247118" watchObservedRunningTime="2026-06-08 18:15:50.464695052 +0000 UTC m=+24.625531744" Jun 08 18:15:51.332321 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:51.332289 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:51.332475 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:51.332408 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:51.448179 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:51.448150 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:15:51.448703 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:51.448533 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"2c567316bb28a95d8a68a110939debc9588a89138fedab1b4de4c837a41f9d2d"} Jun 08 18:15:52.332042 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:52.332014 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:52.332207 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:52.332138 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:52.332207 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:52.332197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:52.332324 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:52.332294 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:52.353069 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:52.353045 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:52.353657 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:52.353631 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:52.451440 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:52.451418 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vl77q" Jun 08 18:15:53.331993 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:53.331824 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:53.332166 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:53.332065 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:53.455590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:53.455572 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:15:53.456099 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:53.455973 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"12073cc39d8ef17e24a5c7aef5927074c34be0b90e849388eb2bdd85d3099ebe"} Jun 08 18:15:53.456212 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:53.456196 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:53.456379 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:53.456366 2560 scope.go:117] "RemoveContainer" containerID="247173a5aafc51241fbb6d1965ad0de38133fc5669c5c5adb9424a9233f47998" Jun 08 18:15:53.458763 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:53.458742 2560 generic.go:356] "Generic (PLEG): container finished" podID="d1540b0a-c591-4239-8ac2-41fd39c3eae3" containerID="0d83818096e4e3d316d42c629db30f9358d550f4ea7e1fa1f3c1b04e257a5436" exitCode=0 Jun 08 18:15:53.458866 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:53.458777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26l7d" event={"ID":"d1540b0a-c591-4239-8ac2-41fd39c3eae3","Type":"ContainerDied","Data":"0d83818096e4e3d316d42c629db30f9358d550f4ea7e1fa1f3c1b04e257a5436"} Jun 08 18:15:53.472635 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:53.472621 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:54.332087 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.332060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:54.332193 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:54.332158 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:54.332239 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.332213 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:54.332319 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:54.332293 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:54.465348 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.465285 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:15:54.465731 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.465658 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" event={"ID":"3985b2b9-e6b3-4430-ae83-217a9afde35f","Type":"ContainerStarted","Data":"4c9a71424fd0a0477cfc79c4668f3b0b41fd8486cdec092bcfd68e092344fca3"} Jun 08 18:15:54.465870 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.465828 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:54.465944 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.465916 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:54.468235 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.468210 2560 generic.go:356] "Generic (PLEG): container finished" podID="d1540b0a-c591-4239-8ac2-41fd39c3eae3" containerID="5fcc462f8ac86f39bf63b060241fe3619846497e51b91980dda679765aa8a8ba" exitCode=0 Jun 08 18:15:54.468338 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.468261 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26l7d" event={"ID":"d1540b0a-c591-4239-8ac2-41fd39c3eae3","Type":"ContainerDied","Data":"5fcc462f8ac86f39bf63b060241fe3619846497e51b91980dda679765aa8a8ba"} Jun 08 18:15:54.481407 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.481384 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:15:54.495553 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:54.495504 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" podStartSLOduration=9.651860809 podStartE2EDuration="28.495491396s" totalImagesPullingTime="18.843630587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:15:54.495368742 +0000 UTC m=+28.656205413" watchObservedRunningTime="2026-06-08 18:15:54.495491396 +0000 UTC m=+28.656328055" Jun 08 18:15:55.104377 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:55.104218 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lbhcf"] Jun 08 18:15:55.104521 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:55.104507 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:55.104632 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:55.104613 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:55.106686 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:55.106664 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p99xb"] Jun 08 18:15:55.106760 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:55.106746 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:55.106827 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:55.106811 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:55.114089 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:55.114069 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nk59b"] Jun 08 18:15:55.114186 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:55.114174 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:55.114290 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:55.114274 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:55.472683 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:55.472649 2560 generic.go:356] "Generic (PLEG): container finished" podID="d1540b0a-c591-4239-8ac2-41fd39c3eae3" containerID="a6bd17de57e9778c87d3e9c471f61f60d2c0f1745cbeedb1b8f82cde190bfbd5" exitCode=0 Jun 08 18:15:55.473211 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:55.472737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26l7d" event={"ID":"d1540b0a-c591-4239-8ac2-41fd39c3eae3","Type":"ContainerDied","Data":"a6bd17de57e9778c87d3e9c471f61f60d2c0f1745cbeedb1b8f82cde190bfbd5"} Jun 08 18:15:56.332816 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:56.332787 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:56.333008 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:56.332907 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:56.333008 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:56.332987 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:56.333119 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:56.333098 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:57.332614 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:57.332581 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:57.333072 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:57.332732 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:15:58.333946 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:58.333918 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:15:58.334540 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:58.334026 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:15:58.334540 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:58.334373 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:15:58.334540 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:58.334443 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:15:59.332182 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:15:59.332147 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:15:59.332349 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:15:59.332290 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:16:00.031613 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.031436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:16:00.032012 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.031641 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:16:00.032012 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.031582 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:16:00.032012 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.031729 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:32.031713942 +0000 UTC m=+66.192550579 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:16:00.032012 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.031766 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:16:00.032012 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.031823 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret podName:796c82e4-e1bb-436b-9145-8377aed43b06 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:32.031807725 +0000 UTC m=+66.192644361 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret") pod "global-pull-secret-syncer-lbhcf" (UID: "796c82e4-e1bb-436b-9145-8377aed43b06") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:16:00.132137 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.132111 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:16:00.132259 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.132214 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:16:00.132259 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.132230 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:16:00.132259 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.132238 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kr5vn for pod openshift-network-diagnostics/network-check-target-p99xb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:16:00.132369 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.132279 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn podName:9ed73b24-9670-44b6-84f2-9a645841edf2 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:32.132265686 +0000 UTC m=+66.293102322 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kr5vn" (UniqueName: "kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn") pod "network-check-target-p99xb" (UID: "9ed73b24-9670-44b6-84f2-9a645841edf2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:16:00.331911 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.331883 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:16:00.332045 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.331920 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:16:00.332045 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.331987 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbhcf" podUID="796c82e4-e1bb-436b-9145-8377aed43b06" Jun 08 18:16:00.332121 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.332093 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p99xb" podUID="9ed73b24-9670-44b6-84f2-9a645841edf2" Jun 08 18:16:00.487350 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.487260 2560 generic.go:356] "Generic (PLEG): container finished" podID="d1540b0a-c591-4239-8ac2-41fd39c3eae3" containerID="29f5f26ce5324efbc7d88b04ca4e654ac416b7a32a50725bac73cafb5c4c2285" exitCode=0 Jun 08 18:16:00.487350 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.487296 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26l7d" event={"ID":"d1540b0a-c591-4239-8ac2-41fd39c3eae3","Type":"ContainerDied","Data":"29f5f26ce5324efbc7d88b04ca4e654ac416b7a32a50725bac73cafb5c4c2285"} Jun 08 18:16:00.705410 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.705390 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeReady" Jun 08 18:16:00.705554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.705509 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 08 18:16:00.740275 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.740229 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59fc7bd9fc-dnzfd"] Jun 08 18:16:00.751550 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.751523 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.754251 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.754230 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59fc7bd9fc-dnzfd"] Jun 08 18:16:00.754514 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.754311 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 18:16:00.754514 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.754339 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 18:16:00.754514 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.754362 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 18:16:00.754514 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.754364 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fn6zv\"" Jun 08 18:16:00.757973 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.757924 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l7b8f"] Jun 08 18:16:00.759336 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.759275 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 18:16:00.771778 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.771759 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8pb2c"] Jun 08 18:16:00.771931 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.771913 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:00.774356 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.774339 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mkk86\"" Jun 08 18:16:00.774356 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.774350 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 08 18:16:00.774504 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.774493 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 08 18:16:00.784392 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.784376 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l7b8f"] Jun 08 18:16:00.784474 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.784403 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8pb2c"] Jun 08 18:16:00.784474 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.784468 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8pb2c" Jun 08 18:16:00.787063 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.787047 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 08 18:16:00.787250 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.787234 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 08 18:16:00.787336 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.787320 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 08 18:16:00.787394 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.787352 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fsh2v\"" Jun 08 18:16:00.836517 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.836497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-image-registry-private-configuration\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.836620 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.836523 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-installation-pull-secrets\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.836620 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.836541 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.836620 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.836556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f85500a-df52-4957-bd0d-cbb16d196bcf-ca-trust-extracted\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.836620 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.836573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-bound-sa-token\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.836759 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.836707 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsvv4\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-kube-api-access-vsvv4\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.836759 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.836741 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-certificates\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.836829 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.836757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-trusted-ca\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.937829 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.937805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-certificates\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.937955 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.937834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-trusted-ca\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.937955 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.937876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-image-registry-private-configuration\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.937955 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.937919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-installation-pull-secrets\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.937955 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.937947 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.938163 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.937963 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f85500a-df52-4957-bd0d-cbb16d196bcf-ca-trust-extracted\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.938163 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.937995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-bound-sa-token\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.938163 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938023 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k66j\" (UniqueName: \"kubernetes.io/projected/1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93-kube-api-access-7k66j\") pod \"ingress-canary-8pb2c\" (UID: \"1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93\") " pod="openshift-ingress-canary/ingress-canary-8pb2c" Jun 08 18:16:00.938163 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.938045 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:00.938163 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.938058 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:16:00.938163 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:00.938104 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:16:01.438088444 +0000 UTC m=+35.598925079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:16:00.938163 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938143 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld2c2\" (UniqueName: \"kubernetes.io/projected/781da7a9-9004-4d56-897b-de4d48846c9f-kube-api-access-ld2c2\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:00.938512 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93-cert\") pod \"ingress-canary-8pb2c\" (UID: \"1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93\") " pod="openshift-ingress-canary/ingress-canary-8pb2c" Jun 08 18:16:00.938512 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938265 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/781da7a9-9004-4d56-897b-de4d48846c9f-config-volume\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:00.938512 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938288 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/781da7a9-9004-4d56-897b-de4d48846c9f-tmp-dir\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:00.938512 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938319 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:00.938512 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvv4\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-kube-api-access-vsvv4\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.938512 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938387 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f85500a-df52-4957-bd0d-cbb16d196bcf-ca-trust-extracted\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.938792 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.938661 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-certificates\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.939408 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.939387 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-trusted-ca\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.942192 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.942169 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-installation-pull-secrets\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.942192 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.942185 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-image-registry-private-configuration\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.948460 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.948437 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsvv4\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-kube-api-access-vsvv4\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:00.948536 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:00.948440 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-bound-sa-token\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:01.038726 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.038638 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93-cert\") pod \"ingress-canary-8pb2c\" (UID: \"1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93\") " pod="openshift-ingress-canary/ingress-canary-8pb2c" Jun 08 18:16:01.038726 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.038672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/781da7a9-9004-4d56-897b-de4d48846c9f-config-volume\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:01.038726 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.038690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/781da7a9-9004-4d56-897b-de4d48846c9f-tmp-dir\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:01.039251 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.038815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:01.039251 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.038934 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k66j\" (UniqueName: \"kubernetes.io/projected/1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93-kube-api-access-7k66j\") pod \"ingress-canary-8pb2c\" (UID: \"1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93\") " pod="openshift-ingress-canary/ingress-canary-8pb2c" Jun 08 18:16:01.039251 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:01.038952 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:16:01.039251 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.038966 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld2c2\" (UniqueName: \"kubernetes.io/projected/781da7a9-9004-4d56-897b-de4d48846c9f-kube-api-access-ld2c2\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:01.039251 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.039001 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/781da7a9-9004-4d56-897b-de4d48846c9f-tmp-dir\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:01.039251 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:01.039023 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:16:01.539001371 +0000 UTC m=+35.699838021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:16:01.039468 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.039346 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/781da7a9-9004-4d56-897b-de4d48846c9f-config-volume\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:01.041172 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.041155 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93-cert\") pod \"ingress-canary-8pb2c\" (UID: \"1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93\") " pod="openshift-ingress-canary/ingress-canary-8pb2c" Jun 08 18:16:01.047345 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.047322 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld2c2\" (UniqueName: \"kubernetes.io/projected/781da7a9-9004-4d56-897b-de4d48846c9f-kube-api-access-ld2c2\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:01.047465 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.047449 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k66j\" (UniqueName: \"kubernetes.io/projected/1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93-kube-api-access-7k66j\") pod \"ingress-canary-8pb2c\" (UID: \"1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93\") " pod="openshift-ingress-canary/ingress-canary-8pb2c" Jun 08 18:16:01.091691 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.091670 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8pb2c" Jun 08 18:16:01.259387 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.259345 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8pb2c"] Jun 08 18:16:01.263813 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:16:01.263787 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb37a57_1c66_4ca1_8bc3_278f5e0c5f93.slice/crio-b1ce99d987048a6acfc988cc2ce39d6f3d07fae7110abc9f966836154ed44e70 WatchSource:0}: Error finding container b1ce99d987048a6acfc988cc2ce39d6f3d07fae7110abc9f966836154ed44e70: Status 404 returned error can't find the container with id b1ce99d987048a6acfc988cc2ce39d6f3d07fae7110abc9f966836154ed44e70 Jun 08 18:16:01.331713 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.331686 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:16:01.334305 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.334285 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 18:16:01.334414 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.334374 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pfkcm\"" Jun 08 18:16:01.441452 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.441419 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:01.441619 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:01.441576 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:01.441619 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:01.441594 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:16:01.441692 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:01.441651 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:16:02.441634068 +0000 UTC m=+36.602470708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:16:01.492751 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.492718 2560 generic.go:356] "Generic (PLEG): container finished" podID="d1540b0a-c591-4239-8ac2-41fd39c3eae3" containerID="50743b6e8655910a46f495a46699b16235a4365161e58d76e37545e21a7a1197" exitCode=0 Jun 08 18:16:01.492926 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.492795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26l7d" event={"ID":"d1540b0a-c591-4239-8ac2-41fd39c3eae3","Type":"ContainerDied","Data":"50743b6e8655910a46f495a46699b16235a4365161e58d76e37545e21a7a1197"} Jun 08 18:16:01.493938 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.493913 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8pb2c" event={"ID":"1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93","Type":"ContainerStarted","Data":"b1ce99d987048a6acfc988cc2ce39d6f3d07fae7110abc9f966836154ed44e70"} Jun 08 18:16:01.542237 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:01.542211 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:01.542362 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:01.542345 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:16:01.542421 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:01.542412 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:16:02.542393163 +0000 UTC m=+36.703229816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:16:02.332191 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.332020 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:16:02.332711 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.332089 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:16:02.334944 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.334918 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 18:16:02.335060 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.334953 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 18:16:02.335795 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.335780 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 18:16:02.335877 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.335823 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ttrkb\"" Jun 08 18:16:02.448801 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.448765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:02.448987 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:02.448926 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:02.448987 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:02.448944 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:16:02.449099 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:02.449009 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:16:04.44898799 +0000 UTC m=+38.609824625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:16:02.500154 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.500112 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-26l7d" event={"ID":"d1540b0a-c591-4239-8ac2-41fd39c3eae3","Type":"ContainerStarted","Data":"38867c7461b14df37fd7565e75ca8b38b087a5e18ef9b3e8974d98d03792e765"} Jun 08 18:16:02.524386 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.524337 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-26l7d" podStartSLOduration=7.791427225 podStartE2EDuration="36.524322257s" totalImagesPullingTime="28.732895032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:02.523296165 +0000 UTC m=+36.684132823" watchObservedRunningTime="2026-06-08 18:16:02.524322257 +0000 UTC m=+36.685158915" Jun 08 18:16:02.549461 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:02.549437 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:02.549606 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:02.549542 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:16:02.549606 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:02.549604 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:16:04.549586451 +0000 UTC m=+38.710423123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:16:04.462922 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:04.462889 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:04.463441 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:04.462990 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:04.463441 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:04.463002 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:16:04.463441 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:04.463049 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:16:08.463035245 +0000 UTC m=+42.623871881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:16:04.506398 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:04.506367 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8pb2c" event={"ID":"1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93","Type":"ContainerStarted","Data":"cc627542a568fbd87fdf65859e98668d10418582a2e2c0915512a8a1307c6642"} Jun 08 18:16:04.523004 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:04.522964 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8pb2c" podStartSLOduration=2.067428043 podStartE2EDuration="4.522946384s" totalImagesPullingTime="2.455518341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:04.522148825 +0000 UTC m=+38.682985480" watchObservedRunningTime="2026-06-08 18:16:04.522946384 +0000 UTC m=+38.683783036" Jun 08 18:16:04.563707 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:04.563677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:04.563875 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:04.563856 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:16:04.563931 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:04.563922 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:16:08.563908713 +0000 UTC m=+42.724745354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:16:07.441277 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.441247 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g"] Jun 08 18:16:07.473474 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.473447 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g"] Jun 08 18:16:07.473603 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.473539 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.476774 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.476752 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 08 18:16:07.477886 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.477864 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 08 18:16:07.477886 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.477826 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 08 18:16:07.478058 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.477933 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 08 18:16:07.582484 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.582459 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95ffeeb-8e21-4351-bd58-3e881690640f-tmp\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.582577 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.582507 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4nq7\" (UniqueName: \"kubernetes.io/projected/d95ffeeb-8e21-4351-bd58-3e881690640f-kube-api-access-v4nq7\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.582622 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.582600 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d95ffeeb-8e21-4351-bd58-3e881690640f-klusterlet-config\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.683711 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.683689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d95ffeeb-8e21-4351-bd58-3e881690640f-klusterlet-config\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.683769 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.683728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95ffeeb-8e21-4351-bd58-3e881690640f-tmp\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.683769 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.683754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4nq7\" (UniqueName: \"kubernetes.io/projected/d95ffeeb-8e21-4351-bd58-3e881690640f-kube-api-access-v4nq7\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.684105 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.684086 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95ffeeb-8e21-4351-bd58-3e881690640f-tmp\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.687311 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.687285 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d95ffeeb-8e21-4351-bd58-3e881690640f-klusterlet-config\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.691780 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.691726 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4nq7\" (UniqueName: \"kubernetes.io/projected/d95ffeeb-8e21-4351-bd58-3e881690640f-kube-api-access-v4nq7\") pod \"klusterlet-addon-workmgr-77f8887947-7964g\" (UID: \"d95ffeeb-8e21-4351-bd58-3e881690640f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.782725 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.782695 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:07.918342 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:07.918313 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g"] Jun 08 18:16:07.921265 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:16:07.921240 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95ffeeb_8e21_4351_bd58_3e881690640f.slice/crio-11cfa51efb74748d0137e0534d7232826142f27d890574374fa78a13433a4995 WatchSource:0}: Error finding container 11cfa51efb74748d0137e0534d7232826142f27d890574374fa78a13433a4995: Status 404 returned error can't find the container with id 11cfa51efb74748d0137e0534d7232826142f27d890574374fa78a13433a4995 Jun 08 18:16:08.492010 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:08.491981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:08.492392 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:08.492109 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:08.492392 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:08.492120 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:16:08.492392 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:08.492173 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:16:16.492154959 +0000 UTC m=+50.652991594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:16:08.516105 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:08.516079 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" event={"ID":"d95ffeeb-8e21-4351-bd58-3e881690640f","Type":"ContainerStarted","Data":"11cfa51efb74748d0137e0534d7232826142f27d890574374fa78a13433a4995"} Jun 08 18:16:08.592797 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:08.592771 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:08.596146 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:08.593700 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:16:08.596146 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:08.593788 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:16:16.593768449 +0000 UTC m=+50.754605102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:16:12.526624 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:12.526557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" event={"ID":"d95ffeeb-8e21-4351-bd58-3e881690640f","Type":"ContainerStarted","Data":"7312fa43e2aa4da924ba68cc268709eca53d31e204f158b0fa80a24826b62b90"} Jun 08 18:16:12.527027 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:12.526748 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:12.528131 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:12.528111 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:16:12.560410 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:12.560368 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" podStartSLOduration=1.22572327 podStartE2EDuration="5.560354934s" totalImagesPullingTime="4.334631664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:12.544037678 +0000 UTC m=+46.704874337" watchObservedRunningTime="2026-06-08 18:16:12.560354934 +0000 UTC m=+46.721191592" Jun 08 18:16:16.551610 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:16.551578 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:16.555959 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:16.551686 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:16.555959 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:16.551699 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:16:16.555959 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:16.551744 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:16:32.55173077 +0000 UTC m=+66.712567405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:16:16.651930 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:16.651906 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:16.652029 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:16.651995 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:16:16.652075 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:16.652036 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:16:32.652025379 +0000 UTC m=+66.812862015 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:16:26.486214 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:26.486183 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4xn9" Jun 08 18:16:32.060306 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.060275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:16:32.060306 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.060309 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:16:32.063739 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.063721 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 18:16:32.064009 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.063995 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 18:16:32.071429 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:32.071415 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 18:16:32.071475 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:32.071467 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:17:36.071452573 +0000 UTC m=+130.232289209 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : secret "metrics-daemon-secret" not found Jun 08 18:16:32.073616 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.073598 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796c82e4-e1bb-436b-9145-8377aed43b06-original-pull-secret\") pod \"global-pull-secret-syncer-lbhcf\" (UID: \"796c82e4-e1bb-436b-9145-8377aed43b06\") " pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:16:32.161273 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.161248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:16:32.164398 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.164377 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 18:16:32.174430 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.174413 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 18:16:32.184809 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.184787 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr5vn\" (UniqueName: \"kubernetes.io/projected/9ed73b24-9670-44b6-84f2-9a645841edf2-kube-api-access-kr5vn\") pod \"network-check-target-p99xb\" (UID: \"9ed73b24-9670-44b6-84f2-9a645841edf2\") " pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:16:32.341515 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.341439 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbhcf" Jun 08 18:16:32.349401 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.349375 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ttrkb\"" Jun 08 18:16:32.356920 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.356898 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:16:32.478204 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.478175 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lbhcf"] Jun 08 18:16:32.481191 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:16:32.481166 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796c82e4_e1bb_436b_9145_8377aed43b06.slice/crio-43b3974ea356cd7362b32135bdf38691181bef7aea92ff0f4050d519a3a4a18b WatchSource:0}: Error finding container 43b3974ea356cd7362b32135bdf38691181bef7aea92ff0f4050d519a3a4a18b: Status 404 returned error can't find the container with id 43b3974ea356cd7362b32135bdf38691181bef7aea92ff0f4050d519a3a4a18b Jun 08 18:16:32.496484 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.496458 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p99xb"] Jun 08 18:16:32.499577 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:16:32.499556 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed73b24_9670_44b6_84f2_9a645841edf2.slice/crio-ea966762edbcc5bbce7a8a2840f3586cd471e9eb6bfa9b3e282d36a29f116de1 WatchSource:0}: Error finding container ea966762edbcc5bbce7a8a2840f3586cd471e9eb6bfa9b3e282d36a29f116de1: Status 404 returned error can't find the container with id ea966762edbcc5bbce7a8a2840f3586cd471e9eb6bfa9b3e282d36a29f116de1 Jun 08 18:16:32.563941 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.563903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:16:32.564104 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:32.564049 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:32.564104 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:32.564069 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:16:32.564172 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:32.564124 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:17:04.564107327 +0000 UTC m=+98.724943966 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:16:32.571803 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.571772 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p99xb" event={"ID":"9ed73b24-9670-44b6-84f2-9a645841edf2","Type":"ContainerStarted","Data":"ea966762edbcc5bbce7a8a2840f3586cd471e9eb6bfa9b3e282d36a29f116de1"} Jun 08 18:16:32.572797 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.572773 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lbhcf" event={"ID":"796c82e4-e1bb-436b-9145-8377aed43b06","Type":"ContainerStarted","Data":"43b3974ea356cd7362b32135bdf38691181bef7aea92ff0f4050d519a3a4a18b"} Jun 08 18:16:32.665106 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:32.665042 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:16:32.665211 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:32.665154 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:16:32.665211 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:16:32.665210 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:17:04.665197248 +0000 UTC m=+98.826033884 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:16:37.588002 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:37.587957 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p99xb" event={"ID":"9ed73b24-9670-44b6-84f2-9a645841edf2","Type":"ContainerStarted","Data":"e2ceba93e2779ff42c36b01ac9fa531f9262a889340acffdcdd9ef068eab96fe"} Jun 08 18:16:37.588428 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:37.588064 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:16:37.589279 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:37.589254 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lbhcf" event={"ID":"796c82e4-e1bb-436b-9145-8377aed43b06","Type":"ContainerStarted","Data":"c33412c0f8bfaf13d0e879994c9c8d5abac1a5baad2bef313fb14380d4ceef40"} Jun 08 18:16:37.606235 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:37.606189 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p99xb" podStartSLOduration=67.348572464 podStartE2EDuration="1m11.606175797s" totalImagesPullingTime="4.257603333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:37.605659012 +0000 UTC m=+71.766495671" watchObservedRunningTime="2026-06-08 18:16:37.606175797 +0000 UTC m=+71.767012433" Jun 08 18:16:37.621893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:16:37.621831 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lbhcf" podStartSLOduration=67.343384315 podStartE2EDuration="1m11.621818169s" totalImagesPullingTime="4.278433854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:37.621570773 +0000 UTC m=+71.782407432" watchObservedRunningTime="2026-06-08 18:16:37.621818169 +0000 UTC m=+71.782654829" Jun 08 18:17:04.581987 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:17:04.581944 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:17:04.582541 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:17:04.582085 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:17:04.582541 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:17:04.582099 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:17:04.582541 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:17:04.582167 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:18:08.582149055 +0000 UTC m=+162.742985694 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:17:04.682705 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:17:04.682670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:17:04.682912 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:17:04.682797 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:17:04.682912 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:17:04.682893 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:18:08.682873716 +0000 UTC m=+162.843710360 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:17:08.594577 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:17:08.594550 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p99xb" Jun 08 18:17:36.089860 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:17:36.089811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:17:36.090410 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:17:36.089960 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 18:17:36.090410 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:17:36.090034 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:19:38.090017632 +0000 UTC m=+252.250854271 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : secret "metrics-daemon-secret" not found Jun 08 18:18:03.762640 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:18:03.762589 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" podUID="4f85500a-df52-4957-bd0d-cbb16d196bcf" Jun 08 18:18:03.780707 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:18:03.780671 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l7b8f" podUID="781da7a9-9004-4d56-897b-de4d48846c9f" Jun 08 18:18:03.785561 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:03.785541 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:18:03.785686 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:03.785541 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7b8f" Jun 08 18:18:04.340066 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:18:04.340029 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:18:08.605975 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:08.605931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:18:08.606344 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:18:08.606067 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:18:08.606344 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:18:08.606083 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59fc7bd9fc-dnzfd: secret "image-registry-tls" not found Jun 08 18:18:08.606344 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:18:08.606136 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls podName:4f85500a-df52-4957-bd0d-cbb16d196bcf nodeName:}" failed. No retries permitted until 2026-06-08 18:20:10.606120549 +0000 UTC m=+284.766957185 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls") pod "image-registry-59fc7bd9fc-dnzfd" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf") : secret "image-registry-tls" not found Jun 08 18:18:08.706789 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:08.706734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:18:08.706975 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:18:08.706925 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:18:08.707027 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:18:08.706997 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls podName:781da7a9-9004-4d56-897b-de4d48846c9f nodeName:}" failed. No retries permitted until 2026-06-08 18:20:10.706979377 +0000 UTC m=+284.867816016 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls") pod "dns-default-l7b8f" (UID: "781da7a9-9004-4d56-897b-de4d48846c9f") : secret "dns-default-metrics-tls" not found Jun 08 18:18:12.527975 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:12.527929 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" podUID="d95ffeeb-8e21-4351-bd58-3e881690640f" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused" Jun 08 18:18:12.807535 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:12.807453 2560 generic.go:356] "Generic (PLEG): container finished" podID="d95ffeeb-8e21-4351-bd58-3e881690640f" containerID="7312fa43e2aa4da924ba68cc268709eca53d31e204f158b0fa80a24826b62b90" exitCode=1 Jun 08 18:18:12.807535 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:12.807515 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" event={"ID":"d95ffeeb-8e21-4351-bd58-3e881690640f","Type":"ContainerDied","Data":"7312fa43e2aa4da924ba68cc268709eca53d31e204f158b0fa80a24826b62b90"} Jun 08 18:18:12.807835 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:12.807818 2560 scope.go:117] "RemoveContainer" containerID="7312fa43e2aa4da924ba68cc268709eca53d31e204f158b0fa80a24826b62b90" Jun 08 18:18:13.814472 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:13.814394 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" event={"ID":"d95ffeeb-8e21-4351-bd58-3e881690640f","Type":"ContainerStarted","Data":"4c9e46fe76e8d0be6ebb1e25558e0ff4c55debaa32c0186f5044fb42a0ef2021"} Jun 08 18:18:13.814820 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:13.814680 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:18:13.815268 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:13.815247 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:18:19.332242 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:18:19.332195 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:19:27.848076 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.848041 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph"] Jun 08 18:19:27.849902 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.849886 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:27.852726 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.852702 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 08 18:19:27.852861 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.852742 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 08 18:19:27.852931 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.852915 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t6b47\"" Jun 08 18:19:27.853781 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.853761 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:19:27.853889 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.853821 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 08 18:19:27.867864 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.867823 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph"] Jun 08 18:19:27.956925 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.956896 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298"] Jun 08 18:19:27.958786 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.958761 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:27.963350 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:27.963317 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-kube-storage-version-migrator-operator\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="921316a0-b53b-447d-a7a6-e9c0d8b4d208" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:27.963718 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:27.963691 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-kube-storage-version-migrator-operator\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jun 08 18:19:27.963796 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:27.963714 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-storage-version-migrator-operator-dockercfg-9jlbs\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-kube-storage-version-migrator-operator\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9jlbs\"" type="*v1.Secret" Jun 08 18:19:27.963796 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:27.963736 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"config\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-kube-storage-version-migrator-operator\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" type="*v1.ConfigMap" Jun 08 18:19:27.964016 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:27.963997 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-kube-storage-version-migrator-operator\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jun 08 18:19:27.965174 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:27.965153 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"serving-cert\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-kube-storage-version-migrator-operator\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" type="*v1.Secret" Jun 08 18:19:27.967277 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.967261 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff597406-794a-48f7-8407-1b2f081007e8-config\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:27.967344 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.967289 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czg9l\" (UniqueName: \"kubernetes.io/projected/ff597406-794a-48f7-8407-1b2f081007e8-kube-api-access-czg9l\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:27.967344 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.967316 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff597406-794a-48f7-8407-1b2f081007e8-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:27.992387 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:27.992364 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298"] Jun 08 18:19:28.067621 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.067594 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl"] Jun 08 18:19:28.067795 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.067775 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/921316a0-b53b-447d-a7a6-e9c0d8b4d208-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:28.067886 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.067805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/921316a0-b53b-447d-a7a6-e9c0d8b4d208-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:28.067930 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.067902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff597406-794a-48f7-8407-1b2f081007e8-config\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:28.067977 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.067938 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czg9l\" (UniqueName: \"kubernetes.io/projected/ff597406-794a-48f7-8407-1b2f081007e8-kube-api-access-czg9l\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:28.068095 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.068076 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff597406-794a-48f7-8407-1b2f081007e8-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:28.068137 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.068124 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x9pl\" (UniqueName: \"kubernetes.io/projected/921316a0-b53b-447d-a7a6-e9c0d8b4d208-kube-api-access-7x9pl\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:28.068448 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.068432 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff597406-794a-48f7-8407-1b2f081007e8-config\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:28.069433 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.069420 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:28.070599 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.070581 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff597406-794a-48f7-8407-1b2f081007e8-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:28.072528 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.072514 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 08 18:19:28.072583 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.072526 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gjrjq\"" Jun 08 18:19:28.072619 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.072526 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 08 18:19:28.084326 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.084303 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czg9l\" (UniqueName: \"kubernetes.io/projected/ff597406-794a-48f7-8407-1b2f081007e8-kube-api-access-czg9l\") pod \"service-ca-operator-6bc5ff68f5-8b6ph\" (UID: \"ff597406-794a-48f7-8407-1b2f081007e8\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:28.084665 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.084643 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl"] Jun 08 18:19:28.159017 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.158931 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" Jun 08 18:19:28.169037 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.169014 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x9pl\" (UniqueName: \"kubernetes.io/projected/921316a0-b53b-447d-a7a6-e9c0d8b4d208-kube-api-access-7x9pl\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:28.169151 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.169068 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/921316a0-b53b-447d-a7a6-e9c0d8b4d208-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:28.169151 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.169100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:28.169151 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.169128 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/921316a0-b53b-447d-a7a6-e9c0d8b4d208-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:28.169309 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.169230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:28.270427 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.270400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:28.270559 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.270452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:28.270604 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:28.270561 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 18:19:28.270642 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:28.270623 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert podName:ce0b5be3-c42e-4ae3-93d2-18244fb5bf36 nodeName:}" failed. No retries permitted until 2026-06-08 18:19:28.770607318 +0000 UTC m=+242.931443967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-rjsgl" (UID: "ce0b5be3-c42e-4ae3-93d2-18244fb5bf36") : secret "networking-console-plugin-cert" not found Jun 08 18:19:28.271021 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.271001 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:28.284690 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.284666 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph"] Jun 08 18:19:28.287728 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:19:28.287705 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff597406_794a_48f7_8407_1b2f081007e8.slice/crio-9919b30f04f97a24bcfd318204fb268c36b15981086c8c56e0e86be80dde6d3d WatchSource:0}: Error finding container 9919b30f04f97a24bcfd318204fb268c36b15981086c8c56e0e86be80dde6d3d: Status 404 returned error can't find the container with id 9919b30f04f97a24bcfd318204fb268c36b15981086c8c56e0e86be80dde6d3d Jun 08 18:19:28.774542 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.774505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:28.774732 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:28.774631 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 18:19:28.774732 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:28.774696 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert podName:ce0b5be3-c42e-4ae3-93d2-18244fb5bf36 nodeName:}" failed. No retries permitted until 2026-06-08 18:19:29.774682952 +0000 UTC m=+243.935519588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-rjsgl" (UID: "ce0b5be3-c42e-4ae3-93d2-18244fb5bf36") : secret "networking-console-plugin-cert" not found Jun 08 18:19:28.847826 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.847794 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 08 18:19:28.850371 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.850347 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/921316a0-b53b-447d-a7a6-e9c0d8b4d208-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:28.975492 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:28.975462 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 08 18:19:29.016215 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.016180 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" event={"ID":"ff597406-794a-48f7-8407-1b2f081007e8","Type":"ContainerStarted","Data":"9919b30f04f97a24bcfd318204fb268c36b15981086c8c56e0e86be80dde6d3d"} Jun 08 18:19:29.080009 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.079988 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 08 18:19:29.092211 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.092190 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/921316a0-b53b-447d-a7a6-e9c0d8b4d208-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:29.271739 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.271707 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:19:29.280940 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.280910 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x9pl\" (UniqueName: \"kubernetes.io/projected/921316a0-b53b-447d-a7a6-e9c0d8b4d208-kube-api-access-7x9pl\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-v7298\" (UID: \"921316a0-b53b-447d-a7a6-e9c0d8b4d208\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:29.395704 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.395623 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9jlbs\"" Jun 08 18:19:29.466749 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.466719 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" Jun 08 18:19:29.590320 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.590279 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298"] Jun 08 18:19:29.593383 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:19:29.593360 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921316a0_b53b_447d_a7a6_e9c0d8b4d208.slice/crio-b72b0875b6fc4194a849a4039e2def302aad11e71b30f4b65abbe283a5970b0f WatchSource:0}: Error finding container b72b0875b6fc4194a849a4039e2def302aad11e71b30f4b65abbe283a5970b0f: Status 404 returned error can't find the container with id b72b0875b6fc4194a849a4039e2def302aad11e71b30f4b65abbe283a5970b0f Jun 08 18:19:29.781771 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:29.781700 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:29.781943 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:29.781883 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 18:19:29.781992 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:29.781961 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert podName:ce0b5be3-c42e-4ae3-93d2-18244fb5bf36 nodeName:}" failed. No retries permitted until 2026-06-08 18:19:31.781940248 +0000 UTC m=+245.942776889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-rjsgl" (UID: "ce0b5be3-c42e-4ae3-93d2-18244fb5bf36") : secret "networking-console-plugin-cert" not found Jun 08 18:19:30.020731 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:30.020687 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" event={"ID":"921316a0-b53b-447d-a7a6-e9c0d8b4d208","Type":"ContainerStarted","Data":"b72b0875b6fc4194a849a4039e2def302aad11e71b30f4b65abbe283a5970b0f"} Jun 08 18:19:31.024982 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:31.024940 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" event={"ID":"ff597406-794a-48f7-8407-1b2f081007e8","Type":"ContainerStarted","Data":"9e73df18a715eeaa2cbfb9fbf0fd8f0a20ab51f5bbbc8f27cf746d3d49c2e9c2"} Jun 08 18:19:31.046613 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:31.046557 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" podStartSLOduration=2.150144494 podStartE2EDuration="4.046543671s" totalImagesPullingTime="1.896399177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:19:31.044257228 +0000 UTC m=+245.205093885" watchObservedRunningTime="2026-06-08 18:19:31.046543671 +0000 UTC m=+245.207380328" Jun 08 18:19:31.793698 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:31.793665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:31.793900 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:31.793818 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 18:19:31.793970 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:31.793911 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert podName:ce0b5be3-c42e-4ae3-93d2-18244fb5bf36 nodeName:}" failed. No retries permitted until 2026-06-08 18:19:35.793893975 +0000 UTC m=+249.954730611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-rjsgl" (UID: "ce0b5be3-c42e-4ae3-93d2-18244fb5bf36") : secret "networking-console-plugin-cert" not found Jun 08 18:19:33.032415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.032383 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" event={"ID":"921316a0-b53b-447d-a7a6-e9c0d8b4d208","Type":"ContainerStarted","Data":"728cf56870814bbab9e7d5c1f2408d103233b96b9e3097e89d2c2c6da1ab18fe"} Jun 08 18:19:33.052353 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.052300 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" podStartSLOduration=3.622467345 podStartE2EDuration="6.052282976s" totalImagesPullingTime="2.429815631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:19:33.051808601 +0000 UTC m=+247.212645264" watchObservedRunningTime="2026-06-08 18:19:33.052282976 +0000 UTC m=+247.213119635" Jun 08 18:19:33.899952 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.899922 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-9j9sw"] Jun 08 18:19:33.901885 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.901865 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:33.904349 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.904295 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 08 18:19:33.904349 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.904325 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 08 18:19:33.905545 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.905512 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 08 18:19:33.905651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.905558 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2txq7\"" Jun 08 18:19:33.905651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.905519 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 08 18:19:33.905753 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.905687 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-signing-key\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:33.905753 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.905717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phsfd\" (UniqueName: \"kubernetes.io/projected/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-kube-api-access-phsfd\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:33.905832 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.905757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-signing-cabundle\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:33.923933 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:33.923912 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-9j9sw"] Jun 08 18:19:34.006525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:34.006497 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-signing-key\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:34.006649 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:34.006528 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phsfd\" (UniqueName: \"kubernetes.io/projected/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-kube-api-access-phsfd\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:34.006649 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:34.006565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-signing-cabundle\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:34.007239 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:34.007223 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-signing-cabundle\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:34.009156 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:34.009138 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-signing-key\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:34.014810 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:34.014789 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phsfd\" (UniqueName: \"kubernetes.io/projected/ba5d391c-1cd4-4c20-a1eb-48c281e6e604-kube-api-access-phsfd\") pod \"service-ca-7d74d578b8-9j9sw\" (UID: \"ba5d391c-1cd4-4c20-a1eb-48c281e6e604\") " pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:34.211158 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:34.211070 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" Jun 08 18:19:34.335416 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:34.335393 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-9j9sw"] Jun 08 18:19:34.335619 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:19:34.335599 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5d391c_1cd4_4c20_a1eb_48c281e6e604.slice/crio-a2e64015c5fe3028b640496f2e84bbcaa93b2aad84558710033b72e92028c1d4 WatchSource:0}: Error finding container a2e64015c5fe3028b640496f2e84bbcaa93b2aad84558710033b72e92028c1d4: Status 404 returned error can't find the container with id a2e64015c5fe3028b640496f2e84bbcaa93b2aad84558710033b72e92028c1d4 Jun 08 18:19:35.041702 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:35.041668 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" event={"ID":"ba5d391c-1cd4-4c20-a1eb-48c281e6e604","Type":"ContainerStarted","Data":"ca67fc700678d6305341e8c6119aa2868d7004c01d63e1f6dc435d2dee79a087"} Jun 08 18:19:35.041877 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:35.041722 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" event={"ID":"ba5d391c-1cd4-4c20-a1eb-48c281e6e604","Type":"ContainerStarted","Data":"a2e64015c5fe3028b640496f2e84bbcaa93b2aad84558710033b72e92028c1d4"} Jun 08 18:19:35.060591 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:35.060541 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-9j9sw" podStartSLOduration=2.060527102 podStartE2EDuration="2.060527102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:19:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:19:35.060001699 +0000 UTC m=+249.220838358" watchObservedRunningTime="2026-06-08 18:19:35.060527102 +0000 UTC m=+249.221363759" Jun 08 18:19:35.442598 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:35.442570 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mqbnh_02733b0c-c0c5-422b-8d01-de38f405ef33/dns-node-resolver/0.log" Jun 08 18:19:35.820293 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:35.820259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:35.820435 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:35.820380 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 18:19:35.820474 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:35.820438 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert podName:ce0b5be3-c42e-4ae3-93d2-18244fb5bf36 nodeName:}" failed. No retries permitted until 2026-06-08 18:19:43.820420981 +0000 UTC m=+257.981257622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-rjsgl" (UID: "ce0b5be3-c42e-4ae3-93d2-18244fb5bf36") : secret "networking-console-plugin-cert" not found Jun 08 18:19:36.244428 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:36.244346 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8cs4k_c4835af7-f4d0-4820-ba05-41ac0bcf4b66/node-ca/0.log" Jun 08 18:19:37.243358 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:37.243322 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8pb2c_1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93/serve-healthcheck-canary/0.log" Jun 08 18:19:38.137493 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:38.137410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:19:38.137657 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:38.137556 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 18:19:38.137657 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:19:38.137614 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs podName:e5c81386-a373-411b-9eb1-205634bdae58 nodeName:}" failed. No retries permitted until 2026-06-08 18:21:40.137598352 +0000 UTC m=+374.298434987 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs") pod "network-metrics-daemon-nk59b" (UID: "e5c81386-a373-411b-9eb1-205634bdae58") : secret "metrics-daemon-secret" not found Jun 08 18:19:43.881017 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:43.880938 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:43.883442 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:43.883418 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0b5be3-c42e-4ae3-93d2-18244fb5bf36-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-rjsgl\" (UID: \"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:43.982508 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:43.982480 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" Jun 08 18:19:44.113947 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:44.113918 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl"] Jun 08 18:19:44.116904 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:19:44.116874 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0b5be3_c42e_4ae3_93d2_18244fb5bf36.slice/crio-dec0a759da9928bfad332640be283871c82703dd883393eae9447d1ca60ef3f5 WatchSource:0}: Error finding container dec0a759da9928bfad332640be283871c82703dd883393eae9447d1ca60ef3f5: Status 404 returned error can't find the container with id dec0a759da9928bfad332640be283871c82703dd883393eae9447d1ca60ef3f5 Jun 08 18:19:45.072108 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:45.072068 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" event={"ID":"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36","Type":"ContainerStarted","Data":"dec0a759da9928bfad332640be283871c82703dd883393eae9447d1ca60ef3f5"} Jun 08 18:19:46.076131 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:46.076101 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" event={"ID":"ce0b5be3-c42e-4ae3-93d2-18244fb5bf36","Type":"ContainerStarted","Data":"eae3aa60938edcaa7355fcb1e5c5a73c565547054222dd51c6e1f0e3592c9853"} Jun 08 18:19:46.095460 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:19:46.095416 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-rjsgl" podStartSLOduration=16.422652548 podStartE2EDuration="18.095401802s" totalImagesPullingTime="1.672749254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:19:46.094358869 +0000 UTC m=+260.255195524" watchObservedRunningTime="2026-06-08 18:19:46.095401802 +0000 UTC m=+260.256238459" Jun 08 18:20:03.327968 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.327935 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56"] Jun 08 18:20:03.332284 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.332262 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" Jun 08 18:20:03.338538 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.338515 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 08 18:20:03.339636 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.339615 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jt6rg\"" Jun 08 18:20:03.342008 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.341991 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 08 18:20:03.359987 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.359956 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56"] Jun 08 18:20:03.360877 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.360857 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xbf77"] Jun 08 18:20:03.363698 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.363682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.367054 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.367039 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 08 18:20:03.367150 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.367060 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 08 18:20:03.367150 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.367114 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 08 18:20:03.367419 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.367404 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 08 18:20:03.367735 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.367723 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hmprp\"" Jun 08 18:20:03.412160 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.412135 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xbf77"] Jun 08 18:20:03.420419 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.420395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0171afcc-f8cd-4930-8d23-04e7825b2721-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.420538 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.420431 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0171afcc-f8cd-4930-8d23-04e7825b2721-crio-socket\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.420592 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.420570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0171afcc-f8cd-4930-8d23-04e7825b2721-data-volume\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.420660 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.420637 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0171afcc-f8cd-4930-8d23-04e7825b2721-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.420780 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.420684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2w26\" (UniqueName: \"kubernetes.io/projected/ad947046-f104-42fc-83f7-b6bbd26fd565-kube-api-access-j2w26\") pod \"migrator-955ffdf5f-vcw56\" (UID: \"ad947046-f104-42fc-83f7-b6bbd26fd565\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" Jun 08 18:20:03.420780 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.420715 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqxnw\" (UniqueName: \"kubernetes.io/projected/0171afcc-f8cd-4930-8d23-04e7825b2721-kube-api-access-fqxnw\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.521050 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521017 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0171afcc-f8cd-4930-8d23-04e7825b2721-crio-socket\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.521224 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0171afcc-f8cd-4930-8d23-04e7825b2721-data-volume\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.521224 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521118 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0171afcc-f8cd-4930-8d23-04e7825b2721-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.521224 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2w26\" (UniqueName: \"kubernetes.io/projected/ad947046-f104-42fc-83f7-b6bbd26fd565-kube-api-access-j2w26\") pod \"migrator-955ffdf5f-vcw56\" (UID: \"ad947046-f104-42fc-83f7-b6bbd26fd565\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" Jun 08 18:20:03.521224 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521154 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqxnw\" (UniqueName: \"kubernetes.io/projected/0171afcc-f8cd-4930-8d23-04e7825b2721-kube-api-access-fqxnw\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.521224 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0171afcc-f8cd-4930-8d23-04e7825b2721-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.521224 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521151 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0171afcc-f8cd-4930-8d23-04e7825b2721-crio-socket\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.521569 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521505 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0171afcc-f8cd-4930-8d23-04e7825b2721-data-volume\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.521704 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.521688 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0171afcc-f8cd-4930-8d23-04e7825b2721-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.523566 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.523546 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0171afcc-f8cd-4930-8d23-04e7825b2721-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.617116 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.617042 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2w26\" (UniqueName: \"kubernetes.io/projected/ad947046-f104-42fc-83f7-b6bbd26fd565-kube-api-access-j2w26\") pod \"migrator-955ffdf5f-vcw56\" (UID: \"ad947046-f104-42fc-83f7-b6bbd26fd565\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" Jun 08 18:20:03.626020 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.625995 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqxnw\" (UniqueName: \"kubernetes.io/projected/0171afcc-f8cd-4930-8d23-04e7825b2721-kube-api-access-fqxnw\") pod \"insights-runtime-extractor-xbf77\" (UID: \"0171afcc-f8cd-4930-8d23-04e7825b2721\") " pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.642945 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.642921 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" Jun 08 18:20:03.671918 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.671890 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xbf77" Jun 08 18:20:03.840451 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.840420 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xbf77"] Jun 08 18:20:03.840736 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:20:03.840715 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0171afcc_f8cd_4930_8d23_04e7825b2721.slice/crio-492ff824c6febfbcc391e3501ec2138e61936cf54edd3f400ad0c88d0db92925 WatchSource:0}: Error finding container 492ff824c6febfbcc391e3501ec2138e61936cf54edd3f400ad0c88d0db92925: Status 404 returned error can't find the container with id 492ff824c6febfbcc391e3501ec2138e61936cf54edd3f400ad0c88d0db92925 Jun 08 18:20:03.872520 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:03.872490 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56"] Jun 08 18:20:03.875211 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:20:03.875189 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad947046_f104_42fc_83f7_b6bbd26fd565.slice/crio-3014238b208a771644a5022e20bc411d0738abd17d1ae6e79b5833c2920e62bb WatchSource:0}: Error finding container 3014238b208a771644a5022e20bc411d0738abd17d1ae6e79b5833c2920e62bb: Status 404 returned error can't find the container with id 3014238b208a771644a5022e20bc411d0738abd17d1ae6e79b5833c2920e62bb Jun 08 18:20:04.126535 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:04.126448 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xbf77" event={"ID":"0171afcc-f8cd-4930-8d23-04e7825b2721","Type":"ContainerStarted","Data":"2ec017876e0fc2b82b69977ddbd14a4409b0d813fed7bc7a403fe542dc95d4b7"} Jun 08 18:20:04.126535 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:04.126498 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xbf77" event={"ID":"0171afcc-f8cd-4930-8d23-04e7825b2721","Type":"ContainerStarted","Data":"492ff824c6febfbcc391e3501ec2138e61936cf54edd3f400ad0c88d0db92925"} Jun 08 18:20:04.127534 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:04.127512 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" event={"ID":"ad947046-f104-42fc-83f7-b6bbd26fd565","Type":"ContainerStarted","Data":"3014238b208a771644a5022e20bc411d0738abd17d1ae6e79b5833c2920e62bb"} Jun 08 18:20:06.135913 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:06.135877 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xbf77" event={"ID":"0171afcc-f8cd-4930-8d23-04e7825b2721","Type":"ContainerStarted","Data":"cbb168d79089360b02f9520b112c81aa336e802d2cf4ce2f0a44294eb8ea453a"} Jun 08 18:20:06.137913 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:06.137884 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" event={"ID":"ad947046-f104-42fc-83f7-b6bbd26fd565","Type":"ContainerStarted","Data":"3eaa73f170114caf06f6e16c699c279dff3b0821f3928921d84978fc04b1cec6"} Jun 08 18:20:06.138016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:06.137949 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" event={"ID":"ad947046-f104-42fc-83f7-b6bbd26fd565","Type":"ContainerStarted","Data":"c4de906a1935f9081d8a78d6eb0adf256cfa47d38d678deca7c60e6cfe8d6adc"} Jun 08 18:20:06.163535 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:06.163481 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-vcw56" podStartSLOduration=1.7459370170000001 podStartE2EDuration="3.163463432s" totalImagesPullingTime="1.417526415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:20:06.161711996 +0000 UTC m=+280.322548653" watchObservedRunningTime="2026-06-08 18:20:06.163463432 +0000 UTC m=+280.324300089" Jun 08 18:20:06.786476 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:20:06.786436 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" podUID="4f85500a-df52-4957-bd0d-cbb16d196bcf" Jun 08 18:20:06.786476 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:20:06.786436 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l7b8f" podUID="781da7a9-9004-4d56-897b-de4d48846c9f" Jun 08 18:20:07.142745 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:07.142534 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xbf77" event={"ID":"0171afcc-f8cd-4930-8d23-04e7825b2721","Type":"ContainerStarted","Data":"e49b6d7e33b1044c68e4984fcfcb8c60e00bc499a9ea6c36aa8920ae2a7e23d0"} Jun 08 18:20:07.144908 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:07.144886 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:20:07.145389 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:07.145375 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7b8f" Jun 08 18:20:07.172517 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:07.172468 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xbf77" podStartSLOduration=1.54543696 podStartE2EDuration="4.172453657s" totalImagesPullingTime="2.627016697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:20:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:20:07.172205232 +0000 UTC m=+281.333041890" watchObservedRunningTime="2026-06-08 18:20:07.172453657 +0000 UTC m=+281.333290314" Jun 08 18:20:10.680280 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:10.680226 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:20:10.682716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:10.682679 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"image-registry-59fc7bd9fc-dnzfd\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:20:10.748483 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:10.748455 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fn6zv\"" Jun 08 18:20:10.756513 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:10.756496 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:20:10.780753 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:10.780722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:20:10.783082 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:10.783062 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/781da7a9-9004-4d56-897b-de4d48846c9f-metrics-tls\") pod \"dns-default-l7b8f\" (UID: \"781da7a9-9004-4d56-897b-de4d48846c9f\") " pod="openshift-dns/dns-default-l7b8f" Jun 08 18:20:10.896948 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:10.896912 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59fc7bd9fc-dnzfd"] Jun 08 18:20:10.901540 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:20:10.901513 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f85500a_df52_4957_bd0d_cbb16d196bcf.slice/crio-75c6fdbdf28e0fe11851420e53a0dfdf07d1fcd2e5af177b95e8076a8937bb16 WatchSource:0}: Error finding container 75c6fdbdf28e0fe11851420e53a0dfdf07d1fcd2e5af177b95e8076a8937bb16: Status 404 returned error can't find the container with id 75c6fdbdf28e0fe11851420e53a0dfdf07d1fcd2e5af177b95e8076a8937bb16 Jun 08 18:20:11.050066 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:11.050034 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mkk86\"" Jun 08 18:20:11.056497 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:11.056464 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7b8f" Jun 08 18:20:11.156274 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:11.156235 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" event={"ID":"4f85500a-df52-4957-bd0d-cbb16d196bcf","Type":"ContainerStarted","Data":"3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152"} Jun 08 18:20:11.156442 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:11.156282 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" event={"ID":"4f85500a-df52-4957-bd0d-cbb16d196bcf","Type":"ContainerStarted","Data":"75c6fdbdf28e0fe11851420e53a0dfdf07d1fcd2e5af177b95e8076a8937bb16"} Jun 08 18:20:11.156442 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:11.156322 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:20:11.180968 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:11.180920 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" podStartSLOduration=285.180905779 podStartE2EDuration="4m45.180905779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:20:11.1805765 +0000 UTC m=+285.341413159" watchObservedRunningTime="2026-06-08 18:20:11.180905779 +0000 UTC m=+285.341742484" Jun 08 18:20:11.187596 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:11.187530 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l7b8f"] Jun 08 18:20:11.191583 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:20:11.191563 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781da7a9_9004_4d56_897b_de4d48846c9f.slice/crio-f6efb468a6cf45c94657fc9109b860e3f9d619fa0ec276ecfd70b7dda9d1f294 WatchSource:0}: Error finding container f6efb468a6cf45c94657fc9109b860e3f9d619fa0ec276ecfd70b7dda9d1f294: Status 404 returned error can't find the container with id f6efb468a6cf45c94657fc9109b860e3f9d619fa0ec276ecfd70b7dda9d1f294 Jun 08 18:20:12.160904 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:12.160866 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7b8f" event={"ID":"781da7a9-9004-4d56-897b-de4d48846c9f","Type":"ContainerStarted","Data":"f6efb468a6cf45c94657fc9109b860e3f9d619fa0ec276ecfd70b7dda9d1f294"} Jun 08 18:20:13.066365 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.066332 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s5bz8"] Jun 08 18:20:13.069731 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.069709 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.072573 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.072548 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 08 18:20:13.072680 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.072606 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mjvc9\"" Jun 08 18:20:13.072971 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.072945 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 08 18:20:13.073740 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.073511 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 08 18:20:13.073740 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.073626 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 08 18:20:13.073966 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.073746 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 08 18:20:13.073966 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.073874 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 08 18:20:13.198490 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198453 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.198917 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198499 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-root\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.198917 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-textfile\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.198917 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198561 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf63ce1b-9335-49ab-a593-a094552a3d35-metrics-client-ca\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.198917 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198621 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6j4c\" (UniqueName: \"kubernetes.io/projected/cf63ce1b-9335-49ab-a593-a094552a3d35-kube-api-access-w6j4c\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.198917 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198685 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.198917 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198741 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-tls\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.198917 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198788 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-sys\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.198917 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.198817 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-wtmp\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300000 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.299906 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300000 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.299954 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-tls\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300000 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.299985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-sys\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300280 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-wtmp\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300280 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300280 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300091 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-root\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300280 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:20:13.300116 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 08 18:20:13.300280 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:20:13.300206 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-tls podName:cf63ce1b-9335-49ab-a593-a094552a3d35 nodeName:}" failed. No retries permitted until 2026-06-08 18:20:13.800183191 +0000 UTC m=+287.961019846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-tls") pod "node-exporter-s5bz8" (UID: "cf63ce1b-9335-49ab-a593-a094552a3d35") : secret "node-exporter-tls" not found Jun 08 18:20:13.300546 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300402 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-textfile\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300546 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300119 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-textfile\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300546 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300476 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-sys\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300546 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300518 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf63ce1b-9335-49ab-a593-a094552a3d35-metrics-client-ca\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300731 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6j4c\" (UniqueName: \"kubernetes.io/projected/cf63ce1b-9335-49ab-a593-a094552a3d35-kube-api-access-w6j4c\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300731 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300614 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300731 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300631 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-wtmp\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.300731 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.300692 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf63ce1b-9335-49ab-a593-a094552a3d35-root\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.301261 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.301227 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf63ce1b-9335-49ab-a593-a094552a3d35-metrics-client-ca\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.303480 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.303450 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.309658 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.309617 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6j4c\" (UniqueName: \"kubernetes.io/projected/cf63ce1b-9335-49ab-a593-a094552a3d35-kube-api-access-w6j4c\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.805123 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.805045 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-tls\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.807349 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.807328 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf63ce1b-9335-49ab-a593-a094552a3d35-node-exporter-tls\") pod \"node-exporter-s5bz8\" (UID: \"cf63ce1b-9335-49ab-a593-a094552a3d35\") " pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.815506 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.815481 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" podUID="d95ffeeb-8e21-4351-bd58-3e881690640f" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused" Jun 08 18:20:13.982508 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:13.982470 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s5bz8" Jun 08 18:20:13.992585 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:20:13.992552 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf63ce1b_9335_49ab_a593_a094552a3d35.slice/crio-ba25ef4b45d9e67b0b2e29dcbee95c0ff394a0ef197310b9884c1a3a4198c095 WatchSource:0}: Error finding container ba25ef4b45d9e67b0b2e29dcbee95c0ff394a0ef197310b9884c1a3a4198c095: Status 404 returned error can't find the container with id ba25ef4b45d9e67b0b2e29dcbee95c0ff394a0ef197310b9884c1a3a4198c095 Jun 08 18:20:14.170899 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.170864 2560 generic.go:356] "Generic (PLEG): container finished" podID="d95ffeeb-8e21-4351-bd58-3e881690640f" containerID="4c9e46fe76e8d0be6ebb1e25558e0ff4c55debaa32c0186f5044fb42a0ef2021" exitCode=1 Jun 08 18:20:14.170899 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.170882 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" event={"ID":"d95ffeeb-8e21-4351-bd58-3e881690640f","Type":"ContainerDied","Data":"4c9e46fe76e8d0be6ebb1e25558e0ff4c55debaa32c0186f5044fb42a0ef2021"} Jun 08 18:20:14.171163 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.170952 2560 scope.go:117] "RemoveContainer" containerID="7312fa43e2aa4da924ba68cc268709eca53d31e204f158b0fa80a24826b62b90" Jun 08 18:20:14.171312 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.171295 2560 scope.go:117] "RemoveContainer" containerID="4c9e46fe76e8d0be6ebb1e25558e0ff4c55debaa32c0186f5044fb42a0ef2021" Jun 08 18:20:14.171532 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:20:14.171498 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-77f8887947-7964g_open-cluster-management-agent-addon(d95ffeeb-8e21-4351-bd58-3e881690640f)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" podUID="d95ffeeb-8e21-4351-bd58-3e881690640f" Jun 08 18:20:14.172905 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.172877 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7b8f" event={"ID":"781da7a9-9004-4d56-897b-de4d48846c9f","Type":"ContainerStarted","Data":"584837692d9a0d789840c79afcfb5dfd2e03a064f38b4c3f43b97cf3de556491"} Jun 08 18:20:14.173005 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.172934 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7b8f" event={"ID":"781da7a9-9004-4d56-897b-de4d48846c9f","Type":"ContainerStarted","Data":"56d364197c3b81dbeb6fcd53e13ba90de0a572b5e3f53a264292f860f51a3301"} Jun 08 18:20:14.173005 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.172993 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l7b8f" Jun 08 18:20:14.174119 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.174098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5bz8" event={"ID":"cf63ce1b-9335-49ab-a593-a094552a3d35","Type":"ContainerStarted","Data":"ba25ef4b45d9e67b0b2e29dcbee95c0ff394a0ef197310b9884c1a3a4198c095"} Jun 08 18:20:14.206956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:14.206906 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l7b8f" podStartSLOduration=251.898765646 podStartE2EDuration="4m14.206893016s" totalImagesPullingTime="2.30812737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:20:14.206188792 +0000 UTC m=+288.367025450" watchObservedRunningTime="2026-06-08 18:20:14.206893016 +0000 UTC m=+288.367729674" Jun 08 18:20:15.179587 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:15.179504 2560 generic.go:356] "Generic (PLEG): container finished" podID="cf63ce1b-9335-49ab-a593-a094552a3d35" containerID="a0286d927fc8699e2b057445e11e49f3f0f95e6c7d41df4041cbab4307e8ef33" exitCode=0 Jun 08 18:20:15.179752 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:15.179588 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5bz8" event={"ID":"cf63ce1b-9335-49ab-a593-a094552a3d35","Type":"ContainerDied","Data":"a0286d927fc8699e2b057445e11e49f3f0f95e6c7d41df4041cbab4307e8ef33"} Jun 08 18:20:16.154095 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.154065 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc"] Jun 08 18:20:16.157417 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.157400 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.162935 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.160607 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 08 18:20:16.162935 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.160637 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 08 18:20:16.162935 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.160830 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bdhbom0h43ej3\"" Jun 08 18:20:16.162935 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.160924 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 08 18:20:16.162935 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.161143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 08 18:20:16.162935 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.161380 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gj78r\"" Jun 08 18:20:16.162935 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.161577 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 08 18:20:16.169862 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.169818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc"] Jun 08 18:20:16.188000 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.187974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5bz8" event={"ID":"cf63ce1b-9335-49ab-a593-a094552a3d35","Type":"ContainerStarted","Data":"063e365bdc1f8ced6b999fb0131b33096fd3d7b0364cd4ea73127e11eec48f9c"} Jun 08 18:20:16.188096 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.188030 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5bz8" event={"ID":"cf63ce1b-9335-49ab-a593-a094552a3d35","Type":"ContainerStarted","Data":"56b115f62a82873233ce07a3cbf27495a7c61fcb2ff7fb5ce895c9633caf49ab"} Jun 08 18:20:16.214302 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.214260 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s5bz8" podStartSLOduration=2.257263002 podStartE2EDuration="3.214248269s" totalImagesPullingTime="887.149286ms" totalInitContainerRuntime="69.835981ms" isStatefulPod=true podCreationTimestamp="2026-06-08 18:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:20:16.212779773 +0000 UTC m=+290.373616432" watchObservedRunningTime="2026-06-08 18:20:16.214248269 +0000 UTC m=+290.375084926" Jun 08 18:20:16.226720 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.226693 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be6b919-a677-4ab5-b8c7-11d35f463b79-metrics-client-ca\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.226799 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.226739 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-grpc-tls\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.226799 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.226781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l24q9\" (UniqueName: \"kubernetes.io/projected/3be6b919-a677-4ab5-b8c7-11d35f463b79-kube-api-access-l24q9\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.226915 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.226810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.226915 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.226864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.226915 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.226900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.227014 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.226926 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-tls\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.227014 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.226966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.327816 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.327792 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be6b919-a677-4ab5-b8c7-11d35f463b79-metrics-client-ca\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.327942 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.327834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-grpc-tls\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.327942 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.327885 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l24q9\" (UniqueName: \"kubernetes.io/projected/3be6b919-a677-4ab5-b8c7-11d35f463b79-kube-api-access-l24q9\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.328047 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.327936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.328047 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.327999 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.328151 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.328062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.328151 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.328093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-tls\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.328151 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.328129 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.328620 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.328600 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be6b919-a677-4ab5-b8c7-11d35f463b79-metrics-client-ca\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.331405 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.331254 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.331405 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.331337 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.331405 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.331362 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.331611 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.331492 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-grpc-tls\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.331818 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.331787 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-tls\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.332097 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.332075 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3be6b919-a677-4ab5-b8c7-11d35f463b79-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.336486 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.336463 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l24q9\" (UniqueName: \"kubernetes.io/projected/3be6b919-a677-4ab5-b8c7-11d35f463b79-kube-api-access-l24q9\") pod \"thanos-querier-6dbb7db6ff-rrxhc\" (UID: \"3be6b919-a677-4ab5-b8c7-11d35f463b79\") " pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.471122 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.471045 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:16.610076 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:16.609993 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc"] Jun 08 18:20:16.612021 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:20:16.611998 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be6b919_a677_4ab5_b8c7_11d35f463b79.slice/crio-8bc53d53e08dc6fca4442545c0c38fa94ed69acba37f97d6d15ba3acdabaab7e WatchSource:0}: Error finding container 8bc53d53e08dc6fca4442545c0c38fa94ed69acba37f97d6d15ba3acdabaab7e: Status 404 returned error can't find the container with id 8bc53d53e08dc6fca4442545c0c38fa94ed69acba37f97d6d15ba3acdabaab7e Jun 08 18:20:17.193785 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:17.193741 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" event={"ID":"3be6b919-a677-4ab5-b8c7-11d35f463b79","Type":"ContainerStarted","Data":"8bc53d53e08dc6fca4442545c0c38fa94ed69acba37f97d6d15ba3acdabaab7e"} Jun 08 18:20:17.783770 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:17.783741 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:20:17.784134 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:17.784118 2560 scope.go:117] "RemoveContainer" containerID="4c9e46fe76e8d0be6ebb1e25558e0ff4c55debaa32c0186f5044fb42a0ef2021" Jun 08 18:20:17.784288 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:20:17.784270 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-77f8887947-7964g_open-cluster-management-agent-addon(d95ffeeb-8e21-4351-bd58-3e881690640f)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" podUID="d95ffeeb-8e21-4351-bd58-3e881690640f" Jun 08 18:20:19.202688 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.202650 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" event={"ID":"3be6b919-a677-4ab5-b8c7-11d35f463b79","Type":"ContainerStarted","Data":"8bfbefc1215dfdc264a9bdc296d9df38ab421aa00bcb75d2e681986a49d82b4c"} Jun 08 18:20:19.203063 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.202717 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" event={"ID":"3be6b919-a677-4ab5-b8c7-11d35f463b79","Type":"ContainerStarted","Data":"2d004cb07faf2556f7d83d2c11e7a935ed5bf8ae3c25039848a325831220105f"} Jun 08 18:20:19.203063 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.202743 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" event={"ID":"3be6b919-a677-4ab5-b8c7-11d35f463b79","Type":"ContainerStarted","Data":"16cd27c587a0a078a2b3389f0caaf3542dc8e79e01cb94a56163b7f6b25e6366"} Jun 08 18:20:19.303029 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.302995 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:20:19.306735 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.306715 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.311045 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.311024 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 18:20:19.311260 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.311241 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 18:20:19.311460 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.311445 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 18:20:19.311460 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.311454 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 18:20:19.311622 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.311543 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 18:20:19.311822 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.311793 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tmkwh\"" Jun 08 18:20:19.311822 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.311794 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 18:20:19.312057 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.311954 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 18:20:19.312331 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.312310 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 18:20:19.312455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.312432 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 18:20:19.312614 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.312600 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 18:20:19.312683 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.312666 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 18:20:19.312783 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.312767 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 18:20:19.313112 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.313090 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 18:20:19.313671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.313649 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 18:20:19.313943 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.313882 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-46lvq8f03cst3\"" Jun 08 18:20:19.316376 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.316355 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 18:20:19.330324 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.330302 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:20:19.452827 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.452758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.452827 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.452791 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.452827 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.452811 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.452829 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-web-config\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.452886 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.452953 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m22vr\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-kube-api-access-m22vr\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453059 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453074 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-config\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453115 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453148 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-config-out\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453315 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.453477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.453336 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554107 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554086 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554201 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554142 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554201 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m22vr\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-kube-api-access-m22vr\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554351 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554351 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554351 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554351 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554312 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-config\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554351 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554346 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554588 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-config-out\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554588 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554411 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554588 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554453 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554588 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554481 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554588 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554491 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554588 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554588 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.554588 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.555016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.555016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.555016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.555016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.555016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-web-config\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.555016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.554984 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.556053 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.555680 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.558308 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.557627 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.558308 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.558022 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.558308 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.558075 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.560037 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.559405 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.560037 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.559971 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.560344 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.560322 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.560790 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.560770 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.560921 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.560898 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.561116 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.561024 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-config\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.561224 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.561193 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.561307 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.561284 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.561436 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.561413 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.561523 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.561463 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.562551 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.562500 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.562987 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.562967 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-web-config\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.563070 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.562974 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-config-out\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.563871 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.563833 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m22vr\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-kube-api-access-m22vr\") pod \"prometheus-k8s-0\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.617771 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.617748 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:19.773658 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:19.773585 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:20:19.775660 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:20:19.775633 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbac6aade_f508_42c9_9fcd_2d9c2db51541.slice/crio-4d65602ef7b160f5c1c7f2a53b063b04dd893bd70ef3b41b87a2bfd3adada9e9 WatchSource:0}: Error finding container 4d65602ef7b160f5c1c7f2a53b063b04dd893bd70ef3b41b87a2bfd3adada9e9: Status 404 returned error can't find the container with id 4d65602ef7b160f5c1c7f2a53b063b04dd893bd70ef3b41b87a2bfd3adada9e9 Jun 08 18:20:20.207173 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:20.207135 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerStarted","Data":"4d65602ef7b160f5c1c7f2a53b063b04dd893bd70ef3b41b87a2bfd3adada9e9"} Jun 08 18:20:20.209685 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:20.209659 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" event={"ID":"3be6b919-a677-4ab5-b8c7-11d35f463b79","Type":"ContainerStarted","Data":"e8138aa101f95409e223f527df4a564efe27ce45a3af1b4f3914415f51b288cd"} Jun 08 18:20:20.209804 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:20.209723 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" event={"ID":"3be6b919-a677-4ab5-b8c7-11d35f463b79","Type":"ContainerStarted","Data":"5ee1ed3f181a471f9330d1b44c84580d48e4ebef3a390c496c48e7f64e35b120"} Jun 08 18:20:20.209804 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:20.209761 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" event={"ID":"3be6b919-a677-4ab5-b8c7-11d35f463b79","Type":"ContainerStarted","Data":"ca96c59cf76da7d5a1d916eee8309cb4c40126aaa1cedfa5d5d4c5887ebec5c0"} Jun 08 18:20:20.210029 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:20.209835 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:20.234786 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:20.234734 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" podStartSLOduration=1.542272962 podStartE2EDuration="4.234719821s" totalImagesPullingTime="2.692446859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:20:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:20:20.233009513 +0000 UTC m=+294.393846192" watchObservedRunningTime="2026-06-08 18:20:20.234719821 +0000 UTC m=+294.395556479" Jun 08 18:20:22.218040 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:22.217998 2560 generic.go:356] "Generic (PLEG): container finished" podID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" exitCode=0 Jun 08 18:20:22.218375 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:22.218049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerDied","Data":"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57"} Jun 08 18:20:22.332619 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:20:22.332585 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nk59b" podUID="e5c81386-a373-411b-9eb1-205634bdae58" Jun 08 18:20:23.814949 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:23.814916 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:20:23.815301 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:23.815287 2560 scope.go:117] "RemoveContainer" containerID="4c9e46fe76e8d0be6ebb1e25558e0ff4c55debaa32c0186f5044fb42a0ef2021" Jun 08 18:20:24.183974 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:24.183885 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l7b8f" Jun 08 18:20:24.228359 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:24.228322 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" event={"ID":"d95ffeeb-8e21-4351-bd58-3e881690640f","Type":"ContainerStarted","Data":"b05cc90cd50f42c7f4904e171764c1480b974c2d41d62508b904df8d582d3f34"} Jun 08 18:20:24.228785 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:24.228709 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:20:24.230331 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:24.230310 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77f8887947-7964g" Jun 08 18:20:25.721233 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:25.714861 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59fc7bd9fc-dnzfd"] Jun 08 18:20:25.727012 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:25.726871 2560 patch_prober.go:28] interesting pod/image-registry-59fc7bd9fc-dnzfd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:20:25.727012 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:25.726935 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" podUID="4f85500a-df52-4957-bd0d-cbb16d196bcf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:20:26.220085 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.220011 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6dbb7db6ff-rrxhc" Jun 08 18:20:26.241424 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.241387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerStarted","Data":"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45"} Jun 08 18:20:26.241574 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.241463 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerStarted","Data":"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b"} Jun 08 18:20:26.241574 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.241490 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerStarted","Data":"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f"} Jun 08 18:20:26.241574 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.241518 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerStarted","Data":"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a"} Jun 08 18:20:26.241694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.241553 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerStarted","Data":"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640"} Jun 08 18:20:26.241694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.241611 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerStarted","Data":"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481"} Jun 08 18:20:26.297278 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.297252 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:20:26.297474 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.297451 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:20:26.301099 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:26.301079 2560 kubelet.go:1648] "Image garbage collection succeeded" Jun 08 18:20:29.618859 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:29.618816 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:20:35.332098 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:35.332067 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:20:35.726009 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:35.725932 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:20:35.751214 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:35.751162 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=11.653839239 podStartE2EDuration="16.75114303s" totalImagesPullingTime="5.076239882s" totalInitContainerRuntime="21.063909ms" isStatefulPod=false podCreationTimestamp="2026-06-08 18:20:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:20:26.289190513 +0000 UTC m=+300.450027195" watchObservedRunningTime="2026-06-08 18:20:35.75114303 +0000 UTC m=+309.911979689" Jun 08 18:20:36.281636 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:36.281604 2560 generic.go:356] "Generic (PLEG): container finished" podID="ff597406-794a-48f7-8407-1b2f081007e8" containerID="9e73df18a715eeaa2cbfb9fbf0fd8f0a20ab51f5bbbc8f27cf746d3d49c2e9c2" exitCode=0 Jun 08 18:20:36.281808 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:36.281680 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" event={"ID":"ff597406-794a-48f7-8407-1b2f081007e8","Type":"ContainerDied","Data":"9e73df18a715eeaa2cbfb9fbf0fd8f0a20ab51f5bbbc8f27cf746d3d49c2e9c2"} Jun 08 18:20:36.282045 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:36.282029 2560 scope.go:117] "RemoveContainer" containerID="9e73df18a715eeaa2cbfb9fbf0fd8f0a20ab51f5bbbc8f27cf746d3d49c2e9c2" Jun 08 18:20:37.288332 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:37.288295 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-8b6ph" event={"ID":"ff597406-794a-48f7-8407-1b2f081007e8","Type":"ContainerStarted","Data":"86ecd3e817a995f37687ed55c43149bac89f5e3bf3841e464003d17d3c460cfe"} Jun 08 18:20:50.742789 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:50.742735 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" podUID="4f85500a-df52-4957-bd0d-cbb16d196bcf" containerName="registry" containerID="cri-o://3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152" gracePeriod=30 Jun 08 18:20:50.985305 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:50.985280 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:20:51.010366 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.010310 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") pod \"4f85500a-df52-4957-bd0d-cbb16d196bcf\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " Jun 08 18:20:51.010366 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.010347 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-certificates\") pod \"4f85500a-df52-4957-bd0d-cbb16d196bcf\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " Jun 08 18:20:51.010494 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.010399 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-trusted-ca\") pod \"4f85500a-df52-4957-bd0d-cbb16d196bcf\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " Jun 08 18:20:51.010494 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.010426 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-bound-sa-token\") pod \"4f85500a-df52-4957-bd0d-cbb16d196bcf\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " Jun 08 18:20:51.010494 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.010446 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f85500a-df52-4957-bd0d-cbb16d196bcf-ca-trust-extracted\") pod \"4f85500a-df52-4957-bd0d-cbb16d196bcf\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " Jun 08 18:20:51.010494 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.010469 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsvv4\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-kube-api-access-vsvv4\") pod \"4f85500a-df52-4957-bd0d-cbb16d196bcf\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " Jun 08 18:20:51.010691 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.010501 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-installation-pull-secrets\") pod \"4f85500a-df52-4957-bd0d-cbb16d196bcf\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " Jun 08 18:20:51.010691 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.010530 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-image-registry-private-configuration\") pod \"4f85500a-df52-4957-bd0d-cbb16d196bcf\" (UID: \"4f85500a-df52-4957-bd0d-cbb16d196bcf\") " Jun 08 18:20:51.011260 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.011114 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4f85500a-df52-4957-bd0d-cbb16d196bcf" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:20:51.011260 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.011222 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4f85500a-df52-4957-bd0d-cbb16d196bcf" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:20:51.013767 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.013737 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4f85500a-df52-4957-bd0d-cbb16d196bcf" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:20:51.014193 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.014124 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4f85500a-df52-4957-bd0d-cbb16d196bcf" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:20:51.014193 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.014125 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-kube-api-access-vsvv4" (OuterVolumeSpecName: "kube-api-access-vsvv4") pod "4f85500a-df52-4957-bd0d-cbb16d196bcf" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf"). InnerVolumeSpecName "kube-api-access-vsvv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:20:51.014193 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.014141 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4f85500a-df52-4957-bd0d-cbb16d196bcf" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:20:51.014450 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.014309 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4f85500a-df52-4957-bd0d-cbb16d196bcf" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:20:51.020486 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.020458 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f85500a-df52-4957-bd0d-cbb16d196bcf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4f85500a-df52-4957-bd0d-cbb16d196bcf" (UID: "4f85500a-df52-4957-bd0d-cbb16d196bcf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:20:51.111235 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.111207 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-certificates\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:20:51.111235 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.111231 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f85500a-df52-4957-bd0d-cbb16d196bcf-trusted-ca\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:20:51.111369 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.111245 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-bound-sa-token\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:20:51.111369 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.111256 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f85500a-df52-4957-bd0d-cbb16d196bcf-ca-trust-extracted\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:20:51.111369 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.111268 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsvv4\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-kube-api-access-vsvv4\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:20:51.111369 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.111280 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-installation-pull-secrets\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:20:51.111369 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.111295 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f85500a-df52-4957-bd0d-cbb16d196bcf-image-registry-private-configuration\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:20:51.111369 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.111309 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f85500a-df52-4957-bd0d-cbb16d196bcf-registry-tls\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:20:51.341056 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.341025 2560 generic.go:356] "Generic (PLEG): container finished" podID="4f85500a-df52-4957-bd0d-cbb16d196bcf" containerID="3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152" exitCode=0 Jun 08 18:20:51.341211 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.341075 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" Jun 08 18:20:51.341211 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.341116 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" event={"ID":"4f85500a-df52-4957-bd0d-cbb16d196bcf","Type":"ContainerDied","Data":"3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152"} Jun 08 18:20:51.341211 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.341160 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59fc7bd9fc-dnzfd" event={"ID":"4f85500a-df52-4957-bd0d-cbb16d196bcf","Type":"ContainerDied","Data":"75c6fdbdf28e0fe11851420e53a0dfdf07d1fcd2e5af177b95e8076a8937bb16"} Jun 08 18:20:51.341211 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.341180 2560 scope.go:117] "RemoveContainer" containerID="3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152" Jun 08 18:20:51.350538 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.350521 2560 scope.go:117] "RemoveContainer" containerID="3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152" Jun 08 18:20:51.350795 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:20:51.350776 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152\": container with ID starting with 3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152 not found: ID does not exist" containerID="3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152" Jun 08 18:20:51.350898 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.350806 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152"} err="failed to get container status \"3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152\": rpc error: code = NotFound desc = could not find container \"3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152\": container with ID starting with 3eae7cef9e731d8fdd5a80fd13c989a4a896600f5fbf1b4dcbcf1d054367f152 not found: ID does not exist" Jun 08 18:20:51.361728 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.361704 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59fc7bd9fc-dnzfd"] Jun 08 18:20:51.366141 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:51.366121 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59fc7bd9fc-dnzfd"] Jun 08 18:20:52.336551 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:52.336511 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f85500a-df52-4957-bd0d-cbb16d196bcf" path="/var/lib/kubelet/pods/4f85500a-df52-4957-bd0d-cbb16d196bcf/volumes" Jun 08 18:20:58.369187 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:58.369093 2560 generic.go:356] "Generic (PLEG): container finished" podID="921316a0-b53b-447d-a7a6-e9c0d8b4d208" containerID="728cf56870814bbab9e7d5c1f2408d103233b96b9e3097e89d2c2c6da1ab18fe" exitCode=0 Jun 08 18:20:58.370361 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:58.369169 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" event={"ID":"921316a0-b53b-447d-a7a6-e9c0d8b4d208","Type":"ContainerDied","Data":"728cf56870814bbab9e7d5c1f2408d103233b96b9e3097e89d2c2c6da1ab18fe"} Jun 08 18:20:58.370361 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:58.370127 2560 scope.go:117] "RemoveContainer" containerID="728cf56870814bbab9e7d5c1f2408d103233b96b9e3097e89d2c2c6da1ab18fe" Jun 08 18:20:59.377931 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:20:59.377118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-v7298" event={"ID":"921316a0-b53b-447d-a7a6-e9c0d8b4d208","Type":"ContainerStarted","Data":"b7fa5eb547e663d94c15e0576ae49fffdaf03c6e6ffa5f613e77e5eb03631d44"} Jun 08 18:21:19.618702 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:19.618616 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:19.639044 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:19.639020 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:20.466408 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:20.466385 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:37.724436 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:37.724405 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:21:37.724882 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:37.724707 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="prometheus" containerID="cri-o://86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" gracePeriod=600 Jun 08 18:21:37.724882 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:37.724725 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy" containerID="cri-o://926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" gracePeriod=600 Jun 08 18:21:37.724882 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:37.724764 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="config-reloader" containerID="cri-o://89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" gracePeriod=600 Jun 08 18:21:37.724882 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:37.724745 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="thanos-sidecar" containerID="cri-o://817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" gracePeriod=600 Jun 08 18:21:37.724882 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:37.724788 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy-web" containerID="cri-o://d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" gracePeriod=600 Jun 08 18:21:37.724882 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:37.724800 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" gracePeriod=600 Jun 08 18:21:37.966106 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:37.966077 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.061105 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061084 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-serving-certs-ca-bundle\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061227 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061125 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-config\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061227 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061150 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-tls\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061227 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061171 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-grpc-tls\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061227 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061201 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-kube-rbac-proxy\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061246 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061279 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-config-out\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061304 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-metrics-client-ca\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061341 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-thanos-prometheus-http-client-file\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061366 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-metrics-client-certs\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061408 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-0\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061433 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-1\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061462 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061490 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-tls-assets\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061501 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061524 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-web-config\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061562 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-trusted-ca-bundle\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061591 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-db\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.061645 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061623 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-kubelet-serving-ca-bundle\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.062139 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061661 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-2\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.062139 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061697 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m22vr\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-kube-api-access-m22vr\") pod \"bac6aade-f508-42c9-9fcd-2d9c2db51541\" (UID: \"bac6aade-f508-42c9-9fcd-2d9c2db51541\") " Jun 08 18:21:38.062139 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.061942 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.062139 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.062055 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:21:38.063091 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.063023 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:21:38.064400 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.064372 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:21:38.064400 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.064378 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:21:38.064721 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.064694 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:21:38.064831 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.064782 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:21:38.064958 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.064927 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-config-out" (OuterVolumeSpecName: "config-out") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:21:38.065035 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.065013 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:21:38.065264 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.065083 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-config" (OuterVolumeSpecName: "config") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.065374 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.065347 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.065456 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.065376 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.065676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.065642 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.066217 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.066185 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.066217 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.066205 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.066412 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.066392 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.066572 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.066555 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-kube-api-access-m22vr" (OuterVolumeSpecName: "kube-api-access-m22vr") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "kube-api-access-m22vr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:21:38.067064 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.067048 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:21:38.067429 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.067412 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.076974 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.076954 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-web-config" (OuterVolumeSpecName: "web-config") pod "bac6aade-f508-42c9-9fcd-2d9c2db51541" (UID: "bac6aade-f508-42c9-9fcd-2d9c2db51541"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:21:38.162463 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162423 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162463 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162460 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-config-out\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162463 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162471 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-metrics-client-ca\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162482 2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162491 2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-metrics-client-certs\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162499 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162509 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162518 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162527 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-tls-assets\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162535 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-web-config\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162542 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162550 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-db\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162558 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162566 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bac6aade-f508-42c9-9fcd-2d9c2db51541-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162574 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m22vr\" (UniqueName: \"kubernetes.io/projected/bac6aade-f508-42c9-9fcd-2d9c2db51541-kube-api-access-m22vr\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162582 2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-config\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162590 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162598 2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-grpc-tls\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.162671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.162606 2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bac6aade-f508-42c9-9fcd-2d9c2db51541-secret-kube-rbac-proxy\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:21:38.517268 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517231 2560 generic.go:356] "Generic (PLEG): container finished" podID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" exitCode=0 Jun 08 18:21:38.517268 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517261 2560 generic.go:356] "Generic (PLEG): container finished" podID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" exitCode=0 Jun 08 18:21:38.517268 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517270 2560 generic.go:356] "Generic (PLEG): container finished" podID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" exitCode=0 Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517279 2560 generic.go:356] "Generic (PLEG): container finished" podID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" exitCode=0 Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517275 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerDied","Data":"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45"} Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517321 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517347 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerDied","Data":"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b"} Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517288 2560 generic.go:356] "Generic (PLEG): container finished" podID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" exitCode=0 Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerDied","Data":"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f"} Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517389 2560 scope.go:117] "RemoveContainer" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517405 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerDied","Data":"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a"} Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517434 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerDied","Data":"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640"} Jun 08 18:21:38.517470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517459 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerDied","Data":"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481"} Jun 08 18:21:38.517827 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517375 2560 generic.go:356] "Generic (PLEG): container finished" podID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" exitCode=0 Jun 08 18:21:38.517827 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.517558 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bac6aade-f508-42c9-9fcd-2d9c2db51541","Type":"ContainerDied","Data":"4d65602ef7b160f5c1c7f2a53b063b04dd893bd70ef3b41b87a2bfd3adada9e9"} Jun 08 18:21:38.525613 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.525596 2560 scope.go:117] "RemoveContainer" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" Jun 08 18:21:38.532320 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.532304 2560 scope.go:117] "RemoveContainer" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" Jun 08 18:21:38.537889 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.537867 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:21:38.539083 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.539064 2560 scope.go:117] "RemoveContainer" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" Jun 08 18:21:38.543037 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.543017 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:21:38.545669 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.545648 2560 scope.go:117] "RemoveContainer" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" Jun 08 18:21:38.551921 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.551898 2560 scope.go:117] "RemoveContainer" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" Jun 08 18:21:38.558677 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.558661 2560 scope.go:117] "RemoveContainer" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" Jun 08 18:21:38.564994 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.564963 2560 scope.go:117] "RemoveContainer" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" Jun 08 18:21:38.565230 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:21:38.565206 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": container with ID starting with d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45 not found: ID does not exist" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" Jun 08 18:21:38.565293 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.565238 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45"} err="failed to get container status \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": rpc error: code = NotFound desc = could not find container \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": container with ID starting with d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45 not found: ID does not exist" Jun 08 18:21:38.565293 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.565255 2560 scope.go:117] "RemoveContainer" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" Jun 08 18:21:38.565497 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:21:38.565481 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": container with ID starting with 926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b not found: ID does not exist" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" Jun 08 18:21:38.565544 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.565502 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b"} err="failed to get container status \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": rpc error: code = NotFound desc = could not find container \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": container with ID starting with 926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b not found: ID does not exist" Jun 08 18:21:38.565544 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.565517 2560 scope.go:117] "RemoveContainer" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" Jun 08 18:21:38.565741 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:21:38.565724 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": container with ID starting with d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f not found: ID does not exist" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" Jun 08 18:21:38.565783 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.565747 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f"} err="failed to get container status \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": rpc error: code = NotFound desc = could not find container \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": container with ID starting with d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f not found: ID does not exist" Jun 08 18:21:38.565783 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.565761 2560 scope.go:117] "RemoveContainer" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" Jun 08 18:21:38.566001 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:21:38.565986 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": container with ID starting with 817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a not found: ID does not exist" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" Jun 08 18:21:38.566044 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566005 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a"} err="failed to get container status \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": rpc error: code = NotFound desc = could not find container \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": container with ID starting with 817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a not found: ID does not exist" Jun 08 18:21:38.566044 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566018 2560 scope.go:117] "RemoveContainer" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" Jun 08 18:21:38.566217 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:21:38.566199 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": container with ID starting with 89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640 not found: ID does not exist" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" Jun 08 18:21:38.566255 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566220 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640"} err="failed to get container status \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": rpc error: code = NotFound desc = could not find container \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": container with ID starting with 89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640 not found: ID does not exist" Jun 08 18:21:38.566255 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566232 2560 scope.go:117] "RemoveContainer" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" Jun 08 18:21:38.566429 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:21:38.566413 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": container with ID starting with 86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481 not found: ID does not exist" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" Jun 08 18:21:38.566473 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566431 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481"} err="failed to get container status \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": rpc error: code = NotFound desc = could not find container \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": container with ID starting with 86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481 not found: ID does not exist" Jun 08 18:21:38.566473 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566443 2560 scope.go:117] "RemoveContainer" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" Jun 08 18:21:38.566641 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:21:38.566626 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": container with ID starting with 20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57 not found: ID does not exist" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" Jun 08 18:21:38.566684 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566643 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57"} err="failed to get container status \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": rpc error: code = NotFound desc = could not find container \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": container with ID starting with 20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57 not found: ID does not exist" Jun 08 18:21:38.566684 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566654 2560 scope.go:117] "RemoveContainer" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" Jun 08 18:21:38.566870 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566833 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45"} err="failed to get container status \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": rpc error: code = NotFound desc = could not find container \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": container with ID starting with d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45 not found: ID does not exist" Jun 08 18:21:38.566870 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.566867 2560 scope.go:117] "RemoveContainer" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" Jun 08 18:21:38.567063 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567049 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b"} err="failed to get container status \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": rpc error: code = NotFound desc = could not find container \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": container with ID starting with 926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b not found: ID does not exist" Jun 08 18:21:38.567110 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567063 2560 scope.go:117] "RemoveContainer" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" Jun 08 18:21:38.567266 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567248 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f"} err="failed to get container status \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": rpc error: code = NotFound desc = could not find container \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": container with ID starting with d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f not found: ID does not exist" Jun 08 18:21:38.567307 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567268 2560 scope.go:117] "RemoveContainer" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" Jun 08 18:21:38.567468 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567453 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a"} err="failed to get container status \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": rpc error: code = NotFound desc = could not find container \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": container with ID starting with 817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a not found: ID does not exist" Jun 08 18:21:38.567506 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567468 2560 scope.go:117] "RemoveContainer" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" Jun 08 18:21:38.567622 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567605 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640"} err="failed to get container status \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": rpc error: code = NotFound desc = could not find container \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": container with ID starting with 89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640 not found: ID does not exist" Jun 08 18:21:38.567669 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567623 2560 scope.go:117] "RemoveContainer" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" Jun 08 18:21:38.567812 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567798 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481"} err="failed to get container status \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": rpc error: code = NotFound desc = could not find container \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": container with ID starting with 86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481 not found: ID does not exist" Jun 08 18:21:38.567887 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.567812 2560 scope.go:117] "RemoveContainer" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" Jun 08 18:21:38.568031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568016 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57"} err="failed to get container status \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": rpc error: code = NotFound desc = could not find container \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": container with ID starting with 20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57 not found: ID does not exist" Jun 08 18:21:38.568031 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568029 2560 scope.go:117] "RemoveContainer" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" Jun 08 18:21:38.568225 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568202 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45"} err="failed to get container status \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": rpc error: code = NotFound desc = could not find container \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": container with ID starting with d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45 not found: ID does not exist" Jun 08 18:21:38.568225 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568226 2560 scope.go:117] "RemoveContainer" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" Jun 08 18:21:38.568440 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568426 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b"} err="failed to get container status \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": rpc error: code = NotFound desc = could not find container \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": container with ID starting with 926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b not found: ID does not exist" Jun 08 18:21:38.568489 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568440 2560 scope.go:117] "RemoveContainer" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" Jun 08 18:21:38.568631 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568614 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f"} err="failed to get container status \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": rpc error: code = NotFound desc = could not find container \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": container with ID starting with d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f not found: ID does not exist" Jun 08 18:21:38.568671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568632 2560 scope.go:117] "RemoveContainer" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" Jun 08 18:21:38.568838 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568820 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a"} err="failed to get container status \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": rpc error: code = NotFound desc = could not find container \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": container with ID starting with 817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a not found: ID does not exist" Jun 08 18:21:38.568900 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.568839 2560 scope.go:117] "RemoveContainer" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" Jun 08 18:21:38.569064 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569049 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640"} err="failed to get container status \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": rpc error: code = NotFound desc = could not find container \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": container with ID starting with 89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640 not found: ID does not exist" Jun 08 18:21:38.569064 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569063 2560 scope.go:117] "RemoveContainer" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" Jun 08 18:21:38.569271 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569253 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481"} err="failed to get container status \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": rpc error: code = NotFound desc = could not find container \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": container with ID starting with 86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481 not found: ID does not exist" Jun 08 18:21:38.569337 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569273 2560 scope.go:117] "RemoveContainer" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" Jun 08 18:21:38.569494 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569478 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57"} err="failed to get container status \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": rpc error: code = NotFound desc = could not find container \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": container with ID starting with 20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57 not found: ID does not exist" Jun 08 18:21:38.569539 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569495 2560 scope.go:117] "RemoveContainer" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" Jun 08 18:21:38.569710 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569690 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45"} err="failed to get container status \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": rpc error: code = NotFound desc = could not find container \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": container with ID starting with d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45 not found: ID does not exist" Jun 08 18:21:38.569751 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569711 2560 scope.go:117] "RemoveContainer" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" Jun 08 18:21:38.569970 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569954 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b"} err="failed to get container status \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": rpc error: code = NotFound desc = could not find container \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": container with ID starting with 926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b not found: ID does not exist" Jun 08 18:21:38.570020 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.569972 2560 scope.go:117] "RemoveContainer" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" Jun 08 18:21:38.570179 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570162 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f"} err="failed to get container status \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": rpc error: code = NotFound desc = could not find container \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": container with ID starting with d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f not found: ID does not exist" Jun 08 18:21:38.570219 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570180 2560 scope.go:117] "RemoveContainer" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" Jun 08 18:21:38.570383 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570366 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a"} err="failed to get container status \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": rpc error: code = NotFound desc = could not find container \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": container with ID starting with 817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a not found: ID does not exist" Jun 08 18:21:38.570449 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570384 2560 scope.go:117] "RemoveContainer" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" Jun 08 18:21:38.570601 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570586 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640"} err="failed to get container status \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": rpc error: code = NotFound desc = could not find container \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": container with ID starting with 89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640 not found: ID does not exist" Jun 08 18:21:38.570652 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570602 2560 scope.go:117] "RemoveContainer" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" Jun 08 18:21:38.570790 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570774 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481"} err="failed to get container status \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": rpc error: code = NotFound desc = could not find container \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": container with ID starting with 86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481 not found: ID does not exist" Jun 08 18:21:38.570832 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570790 2560 scope.go:117] "RemoveContainer" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" Jun 08 18:21:38.571016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.570999 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57"} err="failed to get container status \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": rpc error: code = NotFound desc = could not find container \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": container with ID starting with 20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57 not found: ID does not exist" Jun 08 18:21:38.571060 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571018 2560 scope.go:117] "RemoveContainer" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" Jun 08 18:21:38.571226 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571209 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45"} err="failed to get container status \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": rpc error: code = NotFound desc = could not find container \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": container with ID starting with d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45 not found: ID does not exist" Jun 08 18:21:38.571226 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571226 2560 scope.go:117] "RemoveContainer" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" Jun 08 18:21:38.571505 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571460 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b"} err="failed to get container status \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": rpc error: code = NotFound desc = could not find container \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": container with ID starting with 926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b not found: ID does not exist" Jun 08 18:21:38.571505 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571476 2560 scope.go:117] "RemoveContainer" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" Jun 08 18:21:38.571724 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571699 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f"} err="failed to get container status \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": rpc error: code = NotFound desc = could not find container \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": container with ID starting with d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f not found: ID does not exist" Jun 08 18:21:38.571724 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571724 2560 scope.go:117] "RemoveContainer" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" Jun 08 18:21:38.571985 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571968 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a"} err="failed to get container status \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": rpc error: code = NotFound desc = could not find container \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": container with ID starting with 817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a not found: ID does not exist" Jun 08 18:21:38.571985 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.571985 2560 scope.go:117] "RemoveContainer" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" Jun 08 18:21:38.572326 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.572304 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640"} err="failed to get container status \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": rpc error: code = NotFound desc = could not find container \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": container with ID starting with 89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640 not found: ID does not exist" Jun 08 18:21:38.572326 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.572324 2560 scope.go:117] "RemoveContainer" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" Jun 08 18:21:38.572557 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.572539 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481"} err="failed to get container status \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": rpc error: code = NotFound desc = could not find container \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": container with ID starting with 86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481 not found: ID does not exist" Jun 08 18:21:38.572616 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.572558 2560 scope.go:117] "RemoveContainer" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" Jun 08 18:21:38.572778 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.572759 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57"} err="failed to get container status \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": rpc error: code = NotFound desc = could not find container \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": container with ID starting with 20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57 not found: ID does not exist" Jun 08 18:21:38.572930 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.572779 2560 scope.go:117] "RemoveContainer" containerID="d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45" Jun 08 18:21:38.572930 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.572894 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:21:38.573035 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573011 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45"} err="failed to get container status \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": rpc error: code = NotFound desc = could not find container \"d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45\": container with ID starting with d7451b4e6b07805fa3a5866b882f545e333e961b3bfb854c8c150aec0661ee45 not found: ID does not exist" Jun 08 18:21:38.573035 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573026 2560 scope.go:117] "RemoveContainer" containerID="926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b" Jun 08 18:21:38.573156 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573143 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy-web" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573158 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy-web" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573171 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f85500a-df52-4957-bd0d-cbb16d196bcf" containerName="registry" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573177 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f85500a-df52-4957-bd0d-cbb16d196bcf" containerName="registry" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573183 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="config-reloader" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573188 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="config-reloader" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573195 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="prometheus" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573200 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="prometheus" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573205 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy" Jun 08 18:21:38.573209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573210 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573216 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy-thanos" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573221 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy-thanos" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573220 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b"} err="failed to get container status \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": rpc error: code = NotFound desc = could not find container \"926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b\": container with ID starting with 926e56a8e77c8f3837561845b7905b2808d6c83998ef78d3140f4842ad743d3b not found: ID does not exist" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573246 2560 scope.go:117] "RemoveContainer" containerID="d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573230 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="init-config-reloader" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573290 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="init-config-reloader" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573318 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="thanos-sidecar" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573325 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="thanos-sidecar" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573381 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f85500a-df52-4957-bd0d-cbb16d196bcf" containerName="registry" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573390 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy-web" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573396 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573405 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="thanos-sidecar" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573413 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="config-reloader" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573420 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="kube-rbac-proxy-thanos" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573428 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" containerName="prometheus" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573485 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f"} err="failed to get container status \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": rpc error: code = NotFound desc = could not find container \"d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f\": container with ID starting with d17fdfa2b3ed6c71d960dc4b18f1c5d37bb49ba7490cdd3e08524ff42feddc7f not found: ID does not exist" Jun 08 18:21:38.573525 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573502 2560 scope.go:117] "RemoveContainer" containerID="817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a" Jun 08 18:21:38.574088 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573725 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a"} err="failed to get container status \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": rpc error: code = NotFound desc = could not find container \"817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a\": container with ID starting with 817a114bf8ae6f390c7ab05cb1ec61978a49426ca684c4598c82a285f615ac4a not found: ID does not exist" Jun 08 18:21:38.574088 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573742 2560 scope.go:117] "RemoveContainer" containerID="89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640" Jun 08 18:21:38.574088 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573974 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640"} err="failed to get container status \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": rpc error: code = NotFound desc = could not find container \"89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640\": container with ID starting with 89e1bf364cbda6f8db338c3c314ea75a7f3e8c32ecfb897c3edb496a23309640 not found: ID does not exist" Jun 08 18:21:38.574088 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.573992 2560 scope.go:117] "RemoveContainer" containerID="86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481" Jun 08 18:21:38.574244 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.574195 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481"} err="failed to get container status \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": rpc error: code = NotFound desc = could not find container \"86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481\": container with ID starting with 86e0760b73e9224c902f931e961965a32765ab24abf0e9f2e46a5da1c7d21481 not found: ID does not exist" Jun 08 18:21:38.574244 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.574209 2560 scope.go:117] "RemoveContainer" containerID="20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57" Jun 08 18:21:38.574433 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.574412 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57"} err="failed to get container status \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": rpc error: code = NotFound desc = could not find container \"20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57\": container with ID starting with 20e593ff397efacdefd9f22796339d77360b8f1ca0b5f1c43ef172ce1bfb3d57 not found: ID does not exist" Jun 08 18:21:38.578399 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.578384 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.580943 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.580924 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 18:21:38.581024 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.580926 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tmkwh\"" Jun 08 18:21:38.581343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581191 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 18:21:38.581343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581219 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 18:21:38.581343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581226 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 18:21:38.581343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581219 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 18:21:38.581343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581281 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 18:21:38.581343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581295 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 18:21:38.581343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581224 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 18:21:38.581343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581344 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 18:21:38.581716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581642 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 18:21:38.581716 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581658 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-46lvq8f03cst3\"" Jun 08 18:21:38.581901 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.581886 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 18:21:38.582115 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.582102 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 18:21:38.582199 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.582148 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 18:21:38.589965 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.589945 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 18:21:38.591858 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.591829 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 18:21:38.596544 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.596489 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:21:38.666599 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666575 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666707 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666608 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666707 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666628 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/740056a9-3a16-425d-b597-8bb7a324d936-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666707 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-web-config\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666707 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666693 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666960 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666722 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666960 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666741 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666960 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-config\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666960 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666960 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666888 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666960 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666917 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/740056a9-3a16-425d-b597-8bb7a324d936-config-out\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.666960 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.667299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkvcw\" (UniqueName: \"kubernetes.io/projected/740056a9-3a16-425d-b597-8bb7a324d936-kube-api-access-dkvcw\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.667299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.666999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.667299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.667025 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.667299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.667094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.667299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.667155 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.667299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.667181 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.667299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.667230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.667299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.667258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768120 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768094 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768126 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768177 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768204 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/740056a9-3a16-425d-b597-8bb7a324d936-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768331 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-web-config\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768346 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768363 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768381 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-config\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768411 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768425 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/740056a9-3a16-425d-b597-8bb7a324d936-config-out\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.768457 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768462 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.769363 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkvcw\" (UniqueName: \"kubernetes.io/projected/740056a9-3a16-425d-b597-8bb7a324d936-kube-api-access-dkvcw\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.769363 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.768518 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.769363 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.769142 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.769363 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.769218 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.769634 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.769612 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.769904 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.769883 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.772501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.771393 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.772501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.771558 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.772501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.771609 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.772501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.771706 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.772501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.772399 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.772501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.772412 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.772501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.772474 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.773249 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.773196 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-web-config\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.773249 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.773216 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.773427 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.773406 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-config\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.773475 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.773454 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/740056a9-3a16-425d-b597-8bb7a324d936-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.774133 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.774113 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/740056a9-3a16-425d-b597-8bb7a324d936-config-out\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.774294 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.774279 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.775154 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.775135 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/740056a9-3a16-425d-b597-8bb7a324d936-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.775267 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.775250 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/740056a9-3a16-425d-b597-8bb7a324d936-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.781137 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.781115 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkvcw\" (UniqueName: \"kubernetes.io/projected/740056a9-3a16-425d-b597-8bb7a324d936-kube-api-access-dkvcw\") pod \"prometheus-k8s-0\" (UID: \"740056a9-3a16-425d-b597-8bb7a324d936\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:38.890268 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:38.890214 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:21:39.029159 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:39.029132 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:21:39.032834 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:21:39.032812 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740056a9_3a16_425d_b597_8bb7a324d936.slice/crio-14b7007c5a28a8ce021c9e4c3e08f1d5f372c499232beac4bc62afd05822eff9 WatchSource:0}: Error finding container 14b7007c5a28a8ce021c9e4c3e08f1d5f372c499232beac4bc62afd05822eff9: Status 404 returned error can't find the container with id 14b7007c5a28a8ce021c9e4c3e08f1d5f372c499232beac4bc62afd05822eff9 Jun 08 18:21:39.523151 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:39.523120 2560 generic.go:356] "Generic (PLEG): container finished" podID="740056a9-3a16-425d-b597-8bb7a324d936" containerID="c0a773273ae30ac8e518a48102f610582f0adfe41e5cf329f6ea08cfe0618bc5" exitCode=0 Jun 08 18:21:39.523310 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:39.523181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"740056a9-3a16-425d-b597-8bb7a324d936","Type":"ContainerDied","Data":"c0a773273ae30ac8e518a48102f610582f0adfe41e5cf329f6ea08cfe0618bc5"} Jun 08 18:21:39.523310 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:39.523226 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"740056a9-3a16-425d-b597-8bb7a324d936","Type":"ContainerStarted","Data":"14b7007c5a28a8ce021c9e4c3e08f1d5f372c499232beac4bc62afd05822eff9"} Jun 08 18:21:40.180438 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.180410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:21:40.182709 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.182690 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5c81386-a373-411b-9eb1-205634bdae58-metrics-certs\") pod \"network-metrics-daemon-nk59b\" (UID: \"e5c81386-a373-411b-9eb1-205634bdae58\") " pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:21:40.336794 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.336765 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac6aade-f508-42c9-9fcd-2d9c2db51541" path="/var/lib/kubelet/pods/bac6aade-f508-42c9-9fcd-2d9c2db51541/volumes" Jun 08 18:21:40.435872 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.435806 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pfkcm\"" Jun 08 18:21:40.443717 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.443703 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nk59b" Jun 08 18:21:40.532057 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.532023 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"740056a9-3a16-425d-b597-8bb7a324d936","Type":"ContainerStarted","Data":"7c7b3ad2013d4eea9d0a879e6c0ea0b0cb94736c7226b9b43a40ee168b768ac4"} Jun 08 18:21:40.532154 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.532108 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"740056a9-3a16-425d-b597-8bb7a324d936","Type":"ContainerStarted","Data":"cbdff12a0c834c803b3a8da1191eae418040ac078cc0ab436cff01231aa6dee3"} Jun 08 18:21:40.532189 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.532152 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"740056a9-3a16-425d-b597-8bb7a324d936","Type":"ContainerStarted","Data":"7d809723a822976162053b1bb332e9d7e16aed417198a5959b42503d83469420"} Jun 08 18:21:40.532234 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.532199 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"740056a9-3a16-425d-b597-8bb7a324d936","Type":"ContainerStarted","Data":"19aa14cb4d2e0c040d33c270ecb16d6344dbdbe3ac3041fd5c210b1ac0dfb672"} Jun 08 18:21:40.532269 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.532239 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"740056a9-3a16-425d-b597-8bb7a324d936","Type":"ContainerStarted","Data":"3f5ae4aaff0d2bc4e4fce4141b1e3098845cfbbebd6aff374b5b56ec00d1e7cd"} Jun 08 18:21:40.532312 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.532283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"740056a9-3a16-425d-b597-8bb7a324d936","Type":"ContainerStarted","Data":"03c6cab1283b59cd8c5acf0b0e314e43a8d19dd1fc651c03dfe60adfc4e31c05"} Jun 08 18:21:40.562113 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.562068 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.5398077260000003 podStartE2EDuration="2.562053931s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.246205ms" isStatefulPod=false podCreationTimestamp="2026-06-08 18:21:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:21:40.559902992 +0000 UTC m=+374.720739650" watchObservedRunningTime="2026-06-08 18:21:40.562053931 +0000 UTC m=+374.722890588" Jun 08 18:21:40.574444 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.574414 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nk59b"] Jun 08 18:21:40.577576 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:21:40.577553 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c81386_a373_411b_9eb1_205634bdae58.slice/crio-1fa7c6a902058981a55be38ef495f24c10057958b7507c17bd8d21382c60ce2c WatchSource:0}: Error finding container 1fa7c6a902058981a55be38ef495f24c10057958b7507c17bd8d21382c60ce2c: Status 404 returned error can't find the container with id 1fa7c6a902058981a55be38ef495f24c10057958b7507c17bd8d21382c60ce2c Jun 08 18:21:40.579355 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:40.579339 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 18:21:41.537316 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:41.537286 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nk59b" event={"ID":"e5c81386-a373-411b-9eb1-205634bdae58","Type":"ContainerStarted","Data":"1fa7c6a902058981a55be38ef495f24c10057958b7507c17bd8d21382c60ce2c"} Jun 08 18:21:42.543192 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:42.543154 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nk59b" event={"ID":"e5c81386-a373-411b-9eb1-205634bdae58","Type":"ContainerStarted","Data":"29fe95a7349495bcac28748ff7bee70469240e0b29eee017ec1916c1fa5e32a3"} Jun 08 18:21:42.543658 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:42.543219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nk59b" event={"ID":"e5c81386-a373-411b-9eb1-205634bdae58","Type":"ContainerStarted","Data":"c7754af62a84e450c84733a3e720832cccd8e16229e5d9ce7eb12777b0c12114"} Jun 08 18:21:42.560482 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:42.560398 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nk59b" podStartSLOduration=375.693290646 podStartE2EDuration="6m16.56038085s" totalImagesPullingTime="867.090204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:21:42.559783098 +0000 UTC m=+376.720619757" watchObservedRunningTime="2026-06-08 18:21:42.56038085 +0000 UTC m=+376.721217507" Jun 08 18:21:43.890493 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:21:43.890458 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:22:38.890919 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:22:38.890887 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:22:38.906411 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:22:38.906387 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:22:39.764689 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:22:39.764667 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:24:36.153194 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.151076 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k"] Jun 08 18:24:36.155375 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.155346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:36.158215 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.158188 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 08 18:24:36.158364 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.158188 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 08 18:24:36.159042 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.159024 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-thtdj\"" Jun 08 18:24:36.163973 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.163954 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k"] Jun 08 18:24:36.181590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.181554 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msg4l\" (UniqueName: \"kubernetes.io/projected/1580ebc5-86d8-461b-a822-b4756e15d3b7-kube-api-access-msg4l\") pod \"cert-manager-webhook-7dc76cbb94-xfp5k\" (UID: \"1580ebc5-86d8-461b-a822-b4756e15d3b7\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:36.181684 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.181598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1580ebc5-86d8-461b-a822-b4756e15d3b7-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xfp5k\" (UID: \"1580ebc5-86d8-461b-a822-b4756e15d3b7\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:36.282144 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.282122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1580ebc5-86d8-461b-a822-b4756e15d3b7-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xfp5k\" (UID: \"1580ebc5-86d8-461b-a822-b4756e15d3b7\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:36.282247 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.282178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msg4l\" (UniqueName: \"kubernetes.io/projected/1580ebc5-86d8-461b-a822-b4756e15d3b7-kube-api-access-msg4l\") pod \"cert-manager-webhook-7dc76cbb94-xfp5k\" (UID: \"1580ebc5-86d8-461b-a822-b4756e15d3b7\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:36.300914 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.300890 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1580ebc5-86d8-461b-a822-b4756e15d3b7-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xfp5k\" (UID: \"1580ebc5-86d8-461b-a822-b4756e15d3b7\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:36.301224 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.301204 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msg4l\" (UniqueName: \"kubernetes.io/projected/1580ebc5-86d8-461b-a822-b4756e15d3b7-kube-api-access-msg4l\") pod \"cert-manager-webhook-7dc76cbb94-xfp5k\" (UID: \"1580ebc5-86d8-461b-a822-b4756e15d3b7\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:36.471924 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.471831 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:36.597746 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:36.597718 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k"] Jun 08 18:24:36.600757 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:24:36.600730 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1580ebc5_86d8_461b_a822_b4756e15d3b7.slice/crio-ea58dfb8b8dc4d5fcbb4d6ec5e4bb97205e8b08cc09b39c8dbd823effecf96b4 WatchSource:0}: Error finding container ea58dfb8b8dc4d5fcbb4d6ec5e4bb97205e8b08cc09b39c8dbd823effecf96b4: Status 404 returned error can't find the container with id ea58dfb8b8dc4d5fcbb4d6ec5e4bb97205e8b08cc09b39c8dbd823effecf96b4 Jun 08 18:24:37.165348 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:37.165318 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" event={"ID":"1580ebc5-86d8-461b-a822-b4756e15d3b7","Type":"ContainerStarted","Data":"ea58dfb8b8dc4d5fcbb4d6ec5e4bb97205e8b08cc09b39c8dbd823effecf96b4"} Jun 08 18:24:40.180751 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:40.180717 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" event={"ID":"1580ebc5-86d8-461b-a822-b4756e15d3b7","Type":"ContainerStarted","Data":"06447aa91afd2de0184cc435a05f345c1d3180f72ac2c0c8145b8a16e8315db2"} Jun 08 18:24:40.181206 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:40.180824 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:24:40.210446 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:40.210396 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" podStartSLOduration=1.12382269 podStartE2EDuration="4.210382132s" totalImagesPullingTime="3.086559442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:24:40.208746437 +0000 UTC m=+554.369583107" watchObservedRunningTime="2026-06-08 18:24:40.210382132 +0000 UTC m=+554.371218791" Jun 08 18:24:46.186077 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:24:46.186046 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xfp5k" Jun 08 18:25:07.891601 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:07.891568 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm"] Jun 08 18:25:07.894977 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:07.894960 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:07.898710 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:07.898689 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 08 18:25:07.899079 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:07.899059 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 08 18:25:07.899187 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:07.899170 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-z7kjb\"" Jun 08 18:25:07.899308 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:07.899291 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 08 18:25:07.899651 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:07.899635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 08 18:25:07.916128 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:07.916100 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm"] Jun 08 18:25:08.038616 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.038582 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljcbh\" (UniqueName: \"kubernetes.io/projected/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-kube-api-access-ljcbh\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.038778 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.038624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.038778 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.038659 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-tmp\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.038778 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.038684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-manifests\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.038778 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.038703 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.139223 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.139192 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-tmp\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.139361 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.139231 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-manifests\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.139361 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.139254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.139361 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.139299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljcbh\" (UniqueName: \"kubernetes.io/projected/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-kube-api-access-ljcbh\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.139361 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.139324 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.139655 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.139636 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-manifests\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.141589 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.141569 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-tmp\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.141818 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.141779 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.141946 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.141927 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-webhook-cert\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.147182 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.147161 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljcbh\" (UniqueName: \"kubernetes.io/projected/6f9e5797-8a69-4c8b-af51-6271c1ae2ea3-kube-api-access-ljcbh\") pod \"opendatahub-operator-controller-manager-b4695b4ff-lxvrm\" (UID: \"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3\") " pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.205145 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.205120 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:08.343102 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:08.343080 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm"] Jun 08 18:25:08.345192 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:25:08.345165 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9e5797_8a69_4c8b_af51_6271c1ae2ea3.slice/crio-bada2aad556a1d7af151e8d84904f0a507c733950898fc28fd9abf604185fee9 WatchSource:0}: Error finding container bada2aad556a1d7af151e8d84904f0a507c733950898fc28fd9abf604185fee9: Status 404 returned error can't find the container with id bada2aad556a1d7af151e8d84904f0a507c733950898fc28fd9abf604185fee9 Jun 08 18:25:09.288787 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:09.288744 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" event={"ID":"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3","Type":"ContainerStarted","Data":"bada2aad556a1d7af151e8d84904f0a507c733950898fc28fd9abf604185fee9"} Jun 08 18:25:12.310175 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:12.310145 2560 generic.go:356] "Generic (PLEG): container finished" podID="6f9e5797-8a69-4c8b-af51-6271c1ae2ea3" containerID="009dca63cc9eb66a4982845fe9b8b29e647bbd0a9267ed83416d1aa48f277308" exitCode=0 Jun 08 18:25:12.310501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:12.310182 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" event={"ID":"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3","Type":"ContainerDied","Data":"009dca63cc9eb66a4982845fe9b8b29e647bbd0a9267ed83416d1aa48f277308"} Jun 08 18:25:13.316390 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:13.316355 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" event={"ID":"6f9e5797-8a69-4c8b-af51-6271c1ae2ea3","Type":"ContainerStarted","Data":"0878bc4164de28c143b5451a3506c4a9da436d0ece2f510700d014ede1089913"} Jun 08 18:25:13.316772 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:13.316491 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:13.343490 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:13.343420 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" podStartSLOduration=2.117300304 podStartE2EDuration="6.343401488s" totalImagesPullingTime="3.311958197s" totalInitContainerRuntime="914.142987ms" isStatefulPod=false podCreationTimestamp="2026-06-08 18:25:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:25:13.342151853 +0000 UTC m=+587.502988524" watchObservedRunningTime="2026-06-08 18:25:13.343401488 +0000 UTC m=+587.504238147" Jun 08 18:25:24.322924 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:24.322890 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b4695b4ff-lxvrm" Jun 08 18:25:26.322911 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:26.322883 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:25:26.323368 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:26.323336 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:25:35.648803 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.648764 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr"] Jun 08 18:25:35.652070 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.652055 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.654943 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.654918 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 08 18:25:35.655056 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.655031 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:25:35.655715 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.655701 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 08 18:25:35.655999 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.655981 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 08 18:25:35.656083 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.656034 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5w5nh\"" Jun 08 18:25:35.656083 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.655996 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 08 18:25:35.662091 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.662071 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr"] Jun 08 18:25:35.772777 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.772748 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/768f7c03-86e5-4c54-8370-d6a0a87b0cde-manager-config\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.772945 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.772790 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/768f7c03-86e5-4c54-8370-d6a0a87b0cde-cert\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.772945 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.772807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dgdp\" (UniqueName: \"kubernetes.io/projected/768f7c03-86e5-4c54-8370-d6a0a87b0cde-kube-api-access-8dgdp\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.772945 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.772900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/768f7c03-86e5-4c54-8370-d6a0a87b0cde-metrics-cert\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.873258 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.873232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/768f7c03-86e5-4c54-8370-d6a0a87b0cde-metrics-cert\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.873399 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.873283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/768f7c03-86e5-4c54-8370-d6a0a87b0cde-manager-config\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.873399 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.873307 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/768f7c03-86e5-4c54-8370-d6a0a87b0cde-cert\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.873477 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.873415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dgdp\" (UniqueName: \"kubernetes.io/projected/768f7c03-86e5-4c54-8370-d6a0a87b0cde-kube-api-access-8dgdp\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.873963 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.873938 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/768f7c03-86e5-4c54-8370-d6a0a87b0cde-manager-config\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.876033 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.876011 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/768f7c03-86e5-4c54-8370-d6a0a87b0cde-cert\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.876120 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.876011 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/768f7c03-86e5-4c54-8370-d6a0a87b0cde-metrics-cert\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.882799 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.882779 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dgdp\" (UniqueName: \"kubernetes.io/projected/768f7c03-86e5-4c54-8370-d6a0a87b0cde-kube-api-access-8dgdp\") pod \"lws-controller-manager-5795b6f5cf-q5mcr\" (UID: \"768f7c03-86e5-4c54-8370-d6a0a87b0cde\") " pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:35.961403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:35.961315 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:36.121818 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:36.121783 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr"] Jun 08 18:25:36.127121 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:25:36.127090 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768f7c03_86e5_4c54_8370_d6a0a87b0cde.slice/crio-fb98d6f26f06aa006f7e2e0d92943c8a346e8e0fe0e49e062333b071aa1ebe77 WatchSource:0}: Error finding container fb98d6f26f06aa006f7e2e0d92943c8a346e8e0fe0e49e062333b071aa1ebe77: Status 404 returned error can't find the container with id fb98d6f26f06aa006f7e2e0d92943c8a346e8e0fe0e49e062333b071aa1ebe77 Jun 08 18:25:36.414668 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:36.414630 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" event={"ID":"768f7c03-86e5-4c54-8370-d6a0a87b0cde","Type":"ContainerStarted","Data":"fb98d6f26f06aa006f7e2e0d92943c8a346e8e0fe0e49e062333b071aa1ebe77"} Jun 08 18:25:39.427157 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:39.427123 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" event={"ID":"768f7c03-86e5-4c54-8370-d6a0a87b0cde","Type":"ContainerStarted","Data":"cf1ca5a59504f4376157e75083ffe4cf08211431233f590775fa83b7edd8684d"} Jun 08 18:25:39.427587 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:39.427239 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:25:39.460373 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:39.460327 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" podStartSLOduration=2.016970287 podStartE2EDuration="4.460312479s" totalImagesPullingTime="2.443342192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:25:39.459098106 +0000 UTC m=+613.619934764" watchObservedRunningTime="2026-06-08 18:25:39.460312479 +0000 UTC m=+613.621149137" Jun 08 18:25:50.433823 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:25:50.433735 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5795b6f5cf-q5mcr" Jun 08 18:26:00.255344 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.255311 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w"] Jun 08 18:26:00.260888 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.260868 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.263249 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.263231 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 08 18:26:00.263353 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.263239 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-t6hzl\"" Jun 08 18:26:00.263537 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.263518 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 08 18:26:00.263788 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.263773 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 08 18:26:00.272157 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.272133 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w"] Jun 08 18:26:00.383165 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.383310 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383175 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f04b0cf9-3011-4541-92bf-49510e2699d0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.383310 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383202 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.383310 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.383416 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383324 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.383416 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.383416 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383387 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.383505 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383431 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnczr\" (UniqueName: \"kubernetes.io/projected/f04b0cf9-3011-4541-92bf-49510e2699d0-kube-api-access-rnczr\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.383505 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.383465 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484493 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484493 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484493 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484742 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484742 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484558 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484742 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnczr\" (UniqueName: \"kubernetes.io/projected/f04b0cf9-3011-4541-92bf-49510e2699d0-kube-api-access-rnczr\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484742 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484742 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484636 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484742 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f04b0cf9-3011-4541-92bf-49510e2699d0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.484742 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.484705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.485151 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.485056 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.485234 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.485214 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.485290 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.485235 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.485327 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.485293 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.485877 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.485826 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f04b0cf9-3011-4541-92bf-49510e2699d0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.487259 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.487240 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.487428 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.487411 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.492710 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.492679 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnczr\" (UniqueName: \"kubernetes.io/projected/f04b0cf9-3011-4541-92bf-49510e2699d0-kube-api-access-rnczr\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.493079 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.493061 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f04b0cf9-3011-4541-92bf-49510e2699d0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795lsj7w\" (UID: \"f04b0cf9-3011-4541-92bf-49510e2699d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.573694 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.573663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:00.705491 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:00.705343 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w"] Jun 08 18:26:00.707612 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:26:00.707585 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04b0cf9_3011_4541_92bf_49510e2699d0.slice/crio-8d590d8fa9abc96e9a4f6930918f3fe132df7b89ca044011a528841594a08929 WatchSource:0}: Error finding container 8d590d8fa9abc96e9a4f6930918f3fe132df7b89ca044011a528841594a08929: Status 404 returned error can't find the container with id 8d590d8fa9abc96e9a4f6930918f3fe132df7b89ca044011a528841594a08929 Jun 08 18:26:01.521963 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:01.521929 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" event={"ID":"f04b0cf9-3011-4541-92bf-49510e2699d0","Type":"ContainerStarted","Data":"8d590d8fa9abc96e9a4f6930918f3fe132df7b89ca044011a528841594a08929"} Jun 08 18:26:03.715419 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:03.715378 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 08 18:26:03.715790 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:03.715469 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 08 18:26:03.715790 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:03.715521 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 08 18:26:04.538897 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.538867 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" event={"ID":"f04b0cf9-3011-4541-92bf-49510e2699d0","Type":"ContainerStarted","Data":"d5ad5dc2b8593458f42f5b679b067ef07011075c57674e7e47b190f48d3fbf36"} Jun 08 18:26:04.562233 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.562190 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" podStartSLOduration=1.5567759589999999 podStartE2EDuration="4.56217387s" totalImagesPullingTime="3.005397911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:26:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:26:04.560046404 +0000 UTC m=+638.720883061" watchObservedRunningTime="2026-06-08 18:26:04.56217387 +0000 UTC m=+638.723010528" Jun 08 18:26:04.573969 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.573942 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:04.578164 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.578145 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:04.941233 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.941200 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-84b585c54b-5vkjs"] Jun 08 18:26:04.943543 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.943524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" Jun 08 18:26:04.946268 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.946218 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 08 18:26:04.946268 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.946259 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7786w\"" Jun 08 18:26:04.954821 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:04.954784 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-84b585c54b-5vkjs"] Jun 08 18:26:05.023772 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.023743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/b8c3624d-0ecd-407e-ba87-c45384ad36be-redirect-config\") pod \"dashboard-redirect-84b585c54b-5vkjs\" (UID: \"b8c3624d-0ecd-407e-ba87-c45384ad36be\") " pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" Jun 08 18:26:05.023953 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.023786 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fsl9\" (UniqueName: \"kubernetes.io/projected/b8c3624d-0ecd-407e-ba87-c45384ad36be-kube-api-access-8fsl9\") pod \"dashboard-redirect-84b585c54b-5vkjs\" (UID: \"b8c3624d-0ecd-407e-ba87-c45384ad36be\") " pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" Jun 08 18:26:05.124215 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.124178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/b8c3624d-0ecd-407e-ba87-c45384ad36be-redirect-config\") pod \"dashboard-redirect-84b585c54b-5vkjs\" (UID: \"b8c3624d-0ecd-407e-ba87-c45384ad36be\") " pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" Jun 08 18:26:05.124383 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.124227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fsl9\" (UniqueName: \"kubernetes.io/projected/b8c3624d-0ecd-407e-ba87-c45384ad36be-kube-api-access-8fsl9\") pod \"dashboard-redirect-84b585c54b-5vkjs\" (UID: \"b8c3624d-0ecd-407e-ba87-c45384ad36be\") " pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" Jun 08 18:26:05.124911 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.124882 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/b8c3624d-0ecd-407e-ba87-c45384ad36be-redirect-config\") pod \"dashboard-redirect-84b585c54b-5vkjs\" (UID: \"b8c3624d-0ecd-407e-ba87-c45384ad36be\") " pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" Jun 08 18:26:05.134317 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.134298 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fsl9\" (UniqueName: \"kubernetes.io/projected/b8c3624d-0ecd-407e-ba87-c45384ad36be-kube-api-access-8fsl9\") pod \"dashboard-redirect-84b585c54b-5vkjs\" (UID: \"b8c3624d-0ecd-407e-ba87-c45384ad36be\") " pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" Jun 08 18:26:05.255752 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.255682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" Jun 08 18:26:05.384343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.384320 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-84b585c54b-5vkjs"] Jun 08 18:26:05.386383 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:26:05.386354 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c3624d_0ecd_407e_ba87_c45384ad36be.slice/crio-a5f2b9acd2d8ebaf035e499a7aa6eefe64f6d8e868161636c52bd7a77d6114b4 WatchSource:0}: Error finding container a5f2b9acd2d8ebaf035e499a7aa6eefe64f6d8e868161636c52bd7a77d6114b4: Status 404 returned error can't find the container with id a5f2b9acd2d8ebaf035e499a7aa6eefe64f6d8e868161636c52bd7a77d6114b4 Jun 08 18:26:05.544664 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.544586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" event={"ID":"b8c3624d-0ecd-407e-ba87-c45384ad36be","Type":"ContainerStarted","Data":"a5f2b9acd2d8ebaf035e499a7aa6eefe64f6d8e868161636c52bd7a77d6114b4"} Jun 08 18:26:05.544832 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.544816 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:05.545762 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:05.545742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795lsj7w" Jun 08 18:26:07.554762 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:07.554735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" event={"ID":"b8c3624d-0ecd-407e-ba87-c45384ad36be","Type":"ContainerStarted","Data":"9216c0c7e191559e319ceb86be32ea91382ff78a7293f708967c0d525594c997"} Jun 08 18:26:07.574496 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:07.574453 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-84b585c54b-5vkjs" podStartSLOduration=1.511345748 podStartE2EDuration="3.574438937s" totalImagesPullingTime="2.063093189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:26:07.573500819 +0000 UTC m=+641.734337488" watchObservedRunningTime="2026-06-08 18:26:07.574438937 +0000 UTC m=+641.735275594" Jun 08 18:26:28.730440 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.730407 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vdpmw"] Jun 08 18:26:28.732640 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.732620 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" Jun 08 18:26:28.735250 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.735230 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ztknv\"" Jun 08 18:26:28.736130 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.736114 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 08 18:26:28.736193 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.736143 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 08 18:26:28.747168 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.747141 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vdpmw"] Jun 08 18:26:28.795066 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.795041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjjw5\" (UniqueName: \"kubernetes.io/projected/26a11f07-72e2-4645-9781-0009ce7c71a1-kube-api-access-cjjw5\") pod \"kuadrant-operator-catalog-vdpmw\" (UID: \"26a11f07-72e2-4645-9781-0009ce7c71a1\") " pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" Jun 08 18:26:28.895970 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.895949 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjjw5\" (UniqueName: \"kubernetes.io/projected/26a11f07-72e2-4645-9781-0009ce7c71a1-kube-api-access-cjjw5\") pod \"kuadrant-operator-catalog-vdpmw\" (UID: \"26a11f07-72e2-4645-9781-0009ce7c71a1\") " pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" Jun 08 18:26:28.904624 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:28.904598 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjjw5\" (UniqueName: \"kubernetes.io/projected/26a11f07-72e2-4645-9781-0009ce7c71a1-kube-api-access-cjjw5\") pod \"kuadrant-operator-catalog-vdpmw\" (UID: \"26a11f07-72e2-4645-9781-0009ce7c71a1\") " pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" Jun 08 18:26:29.042582 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.042520 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" Jun 08 18:26:29.071143 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.071118 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vdpmw"] Jun 08 18:26:29.181332 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.181306 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vdpmw"] Jun 08 18:26:29.184940 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:26:29.184904 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a11f07_72e2_4645_9781_0009ce7c71a1.slice/crio-fee20f10b2ccb295346d1c0b9be6631e0aa66286f5258652215cfd0bdc7d325f WatchSource:0}: Error finding container fee20f10b2ccb295346d1c0b9be6631e0aa66286f5258652215cfd0bdc7d325f: Status 404 returned error can't find the container with id fee20f10b2ccb295346d1c0b9be6631e0aa66286f5258652215cfd0bdc7d325f Jun 08 18:26:29.285738 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.285715 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zk8s"] Jun 08 18:26:29.288648 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.288634 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:26:29.301303 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.301272 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zk8s"] Jun 08 18:26:29.399728 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.399693 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc6p6\" (UniqueName: \"kubernetes.io/projected/cb20b223-e9e7-4ec3-92fc-7f7e8fb15873-kube-api-access-jc6p6\") pod \"kuadrant-operator-catalog-2zk8s\" (UID: \"cb20b223-e9e7-4ec3-92fc-7f7e8fb15873\") " pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:26:29.500185 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.500157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc6p6\" (UniqueName: \"kubernetes.io/projected/cb20b223-e9e7-4ec3-92fc-7f7e8fb15873-kube-api-access-jc6p6\") pod \"kuadrant-operator-catalog-2zk8s\" (UID: \"cb20b223-e9e7-4ec3-92fc-7f7e8fb15873\") " pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:26:29.509501 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.509474 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc6p6\" (UniqueName: \"kubernetes.io/projected/cb20b223-e9e7-4ec3-92fc-7f7e8fb15873-kube-api-access-jc6p6\") pod \"kuadrant-operator-catalog-2zk8s\" (UID: \"cb20b223-e9e7-4ec3-92fc-7f7e8fb15873\") " pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:26:29.597070 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.597051 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:26:29.650096 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.650062 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" event={"ID":"26a11f07-72e2-4645-9781-0009ce7c71a1","Type":"ContainerStarted","Data":"fee20f10b2ccb295346d1c0b9be6631e0aa66286f5258652215cfd0bdc7d325f"} Jun 08 18:26:29.730600 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:29.730564 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2zk8s"] Jun 08 18:26:29.733194 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:26:29.733172 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb20b223_e9e7_4ec3_92fc_7f7e8fb15873.slice/crio-a2d35c9ced3c0ed2db32d14c45e6bfcbb8f890f638180877d4583803fc1c3f50 WatchSource:0}: Error finding container a2d35c9ced3c0ed2db32d14c45e6bfcbb8f890f638180877d4583803fc1c3f50: Status 404 returned error can't find the container with id a2d35c9ced3c0ed2db32d14c45e6bfcbb8f890f638180877d4583803fc1c3f50 Jun 08 18:26:30.657091 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:30.657055 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" event={"ID":"cb20b223-e9e7-4ec3-92fc-7f7e8fb15873","Type":"ContainerStarted","Data":"a2d35c9ced3c0ed2db32d14c45e6bfcbb8f890f638180877d4583803fc1c3f50"} Jun 08 18:26:31.663610 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:31.663528 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" event={"ID":"26a11f07-72e2-4645-9781-0009ce7c71a1","Type":"ContainerStarted","Data":"66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef"} Jun 08 18:26:31.663610 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:31.663592 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" podUID="26a11f07-72e2-4645-9781-0009ce7c71a1" containerName="registry-server" containerID="cri-o://66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef" gracePeriod=2 Jun 08 18:26:31.665369 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:31.665345 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" event={"ID":"cb20b223-e9e7-4ec3-92fc-7f7e8fb15873","Type":"ContainerStarted","Data":"9c6df932711bb22977413d8cfc6c59d0156419493b028985b5ba5d8a26b4cf41"} Jun 08 18:26:31.728446 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:31.728398 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" podStartSLOduration=1.5549048829999998 podStartE2EDuration="3.728383385s" totalImagesPullingTime="2.173478502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:26:31.702246189 +0000 UTC m=+665.863082847" watchObservedRunningTime="2026-06-08 18:26:31.728383385 +0000 UTC m=+665.889220044" Jun 08 18:26:31.728704 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:31.728683 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" podStartSLOduration=1.100434077 podStartE2EDuration="2.728678441s" totalImagesPullingTime="1.628244364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:26:31.726829828 +0000 UTC m=+665.887666499" watchObservedRunningTime="2026-06-08 18:26:31.728678441 +0000 UTC m=+665.889515098" Jun 08 18:26:31.904122 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:31.904099 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" Jun 08 18:26:32.022827 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.022730 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjjw5\" (UniqueName: \"kubernetes.io/projected/26a11f07-72e2-4645-9781-0009ce7c71a1-kube-api-access-cjjw5\") pod \"26a11f07-72e2-4645-9781-0009ce7c71a1\" (UID: \"26a11f07-72e2-4645-9781-0009ce7c71a1\") " Jun 08 18:26:32.025026 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.025001 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a11f07-72e2-4645-9781-0009ce7c71a1-kube-api-access-cjjw5" (OuterVolumeSpecName: "kube-api-access-cjjw5") pod "26a11f07-72e2-4645-9781-0009ce7c71a1" (UID: "26a11f07-72e2-4645-9781-0009ce7c71a1"). InnerVolumeSpecName "kube-api-access-cjjw5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:26:32.124087 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.124055 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjjw5\" (UniqueName: \"kubernetes.io/projected/26a11f07-72e2-4645-9781-0009ce7c71a1-kube-api-access-cjjw5\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:26:32.670982 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.670888 2560 generic.go:356] "Generic (PLEG): container finished" podID="26a11f07-72e2-4645-9781-0009ce7c71a1" containerID="66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef" exitCode=0 Jun 08 18:26:32.670982 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.670938 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" Jun 08 18:26:32.670982 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.670970 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" event={"ID":"26a11f07-72e2-4645-9781-0009ce7c71a1","Type":"ContainerDied","Data":"66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef"} Jun 08 18:26:32.671498 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.671018 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vdpmw" event={"ID":"26a11f07-72e2-4645-9781-0009ce7c71a1","Type":"ContainerDied","Data":"fee20f10b2ccb295346d1c0b9be6631e0aa66286f5258652215cfd0bdc7d325f"} Jun 08 18:26:32.671498 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.671038 2560 scope.go:117] "RemoveContainer" containerID="66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef" Jun 08 18:26:32.679880 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.679863 2560 scope.go:117] "RemoveContainer" containerID="66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef" Jun 08 18:26:32.680181 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:26:32.680162 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef\": container with ID starting with 66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef not found: ID does not exist" containerID="66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef" Jun 08 18:26:32.680230 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.680191 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef"} err="failed to get container status \"66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef\": rpc error: code = NotFound desc = could not find container \"66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef\": container with ID starting with 66d078dea6228fe7597cf79a8fb887281bfe85ed99fc5d733e1a12b872dd33ef not found: ID does not exist" Jun 08 18:26:32.687925 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.687903 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vdpmw"] Jun 08 18:26:32.692008 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:32.691989 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vdpmw"] Jun 08 18:26:34.337089 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:34.337053 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a11f07-72e2-4645-9781-0009ce7c71a1" path="/var/lib/kubelet/pods/26a11f07-72e2-4645-9781-0009ce7c71a1/volumes" Jun 08 18:26:39.597517 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:39.597478 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:26:39.598003 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:39.597537 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:26:39.619708 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:39.619686 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:26:39.724205 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:26:39.724181 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2zk8s" Jun 08 18:27:01.279463 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.279427 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5"] Jun 08 18:27:01.279868 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.279809 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a11f07-72e2-4645-9781-0009ce7c71a1" containerName="registry-server" Jun 08 18:27:01.279868 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.279821 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a11f07-72e2-4645-9781-0009ce7c71a1" containerName="registry-server" Jun 08 18:27:01.279946 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.279887 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a11f07-72e2-4645-9781-0009ce7c71a1" containerName="registry-server" Jun 08 18:27:01.284544 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.284528 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" Jun 08 18:27:01.287396 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.287376 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-j2xrv\"" Jun 08 18:27:01.296391 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.296372 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5"] Jun 08 18:27:01.456376 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.456347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7vtt\" (UniqueName: \"kubernetes.io/projected/d45b58d3-44a6-47bf-b27b-028d3760041c-kube-api-access-r7vtt\") pod \"authorino-operator-75cc8d5cb7-wqpb5\" (UID: \"d45b58d3-44a6-47bf-b27b-028d3760041c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" Jun 08 18:27:01.557717 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.557649 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7vtt\" (UniqueName: \"kubernetes.io/projected/d45b58d3-44a6-47bf-b27b-028d3760041c-kube-api-access-r7vtt\") pod \"authorino-operator-75cc8d5cb7-wqpb5\" (UID: \"d45b58d3-44a6-47bf-b27b-028d3760041c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" Jun 08 18:27:01.568485 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.568455 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7vtt\" (UniqueName: \"kubernetes.io/projected/d45b58d3-44a6-47bf-b27b-028d3760041c-kube-api-access-r7vtt\") pod \"authorino-operator-75cc8d5cb7-wqpb5\" (UID: \"d45b58d3-44a6-47bf-b27b-028d3760041c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" Jun 08 18:27:01.594454 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.594430 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" Jun 08 18:27:01.734991 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.734964 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5"] Jun 08 18:27:01.737661 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:27:01.737630 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45b58d3_44a6_47bf_b27b_028d3760041c.slice/crio-c846cac924f9568cc1a2fd52dbf9cdc545b980255238707f7a987b94c20dabf5 WatchSource:0}: Error finding container c846cac924f9568cc1a2fd52dbf9cdc545b980255238707f7a987b94c20dabf5: Status 404 returned error can't find the container with id c846cac924f9568cc1a2fd52dbf9cdc545b980255238707f7a987b94c20dabf5 Jun 08 18:27:01.739766 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.739749 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 18:27:01.792659 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:01.792632 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" event={"ID":"d45b58d3-44a6-47bf-b27b-028d3760041c","Type":"ContainerStarted","Data":"c846cac924f9568cc1a2fd52dbf9cdc545b980255238707f7a987b94c20dabf5"} Jun 08 18:27:04.074393 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.074357 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c"] Jun 08 18:27:04.077624 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.077606 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" Jun 08 18:27:04.080429 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.080409 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hlgcs\"" Jun 08 18:27:04.080545 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.080528 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 08 18:27:04.089220 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.089200 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c"] Jun 08 18:27:04.182397 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.182371 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq5hq\" (UniqueName: \"kubernetes.io/projected/54b0b175-594b-4a50-92f4-c9f46a75913d-kube-api-access-tq5hq\") pod \"dns-operator-controller-manager-864dd78c4c-fcm9c\" (UID: \"54b0b175-594b-4a50-92f4-c9f46a75913d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" Jun 08 18:27:04.283179 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.283148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq5hq\" (UniqueName: \"kubernetes.io/projected/54b0b175-594b-4a50-92f4-c9f46a75913d-kube-api-access-tq5hq\") pod \"dns-operator-controller-manager-864dd78c4c-fcm9c\" (UID: \"54b0b175-594b-4a50-92f4-c9f46a75913d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" Jun 08 18:27:04.294395 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.294368 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq5hq\" (UniqueName: \"kubernetes.io/projected/54b0b175-594b-4a50-92f4-c9f46a75913d-kube-api-access-tq5hq\") pod \"dns-operator-controller-manager-864dd78c4c-fcm9c\" (UID: \"54b0b175-594b-4a50-92f4-c9f46a75913d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" Jun 08 18:27:04.387862 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.387778 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" Jun 08 18:27:04.672637 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.672610 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c"] Jun 08 18:27:04.674870 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:27:04.674818 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b0b175_594b_4a50_92f4_c9f46a75913d.slice/crio-0a6c3b6ff6785c177bcbbc8edaba264a481e32bf979680efe3f1c6e0309a169d WatchSource:0}: Error finding container 0a6c3b6ff6785c177bcbbc8edaba264a481e32bf979680efe3f1c6e0309a169d: Status 404 returned error can't find the container with id 0a6c3b6ff6785c177bcbbc8edaba264a481e32bf979680efe3f1c6e0309a169d Jun 08 18:27:04.808067 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.808035 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" event={"ID":"54b0b175-594b-4a50-92f4-c9f46a75913d","Type":"ContainerStarted","Data":"0a6c3b6ff6785c177bcbbc8edaba264a481e32bf979680efe3f1c6e0309a169d"} Jun 08 18:27:04.809578 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.809554 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" event={"ID":"d45b58d3-44a6-47bf-b27b-028d3760041c","Type":"ContainerStarted","Data":"e6f06a2a473782caa067f98ee5dfd33da6dcca3eae8a8c628162af565fe0f450"} Jun 08 18:27:04.809704 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.809688 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" Jun 08 18:27:04.833708 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:04.833639 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" podStartSLOduration=0.970507244 podStartE2EDuration="3.833611612s" totalImagesPullingTime="2.863104368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:27:04.832326137 +0000 UTC m=+698.993162796" watchObservedRunningTime="2026-06-08 18:27:04.833611612 +0000 UTC m=+698.994448272" Jun 08 18:27:07.826806 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:07.826762 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" event={"ID":"54b0b175-594b-4a50-92f4-c9f46a75913d","Type":"ContainerStarted","Data":"442f86feab0f0978169221846f6c99c086088a2f04dd86f3cb9475e84dece19a"} Jun 08 18:27:07.827226 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:07.826875 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" Jun 08 18:27:07.854419 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:07.854375 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" podStartSLOduration=1.428837317 podStartE2EDuration="3.854359913s" totalImagesPullingTime="2.425522596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:27:07.853095003 +0000 UTC m=+702.013931662" watchObservedRunningTime="2026-06-08 18:27:07.854359913 +0000 UTC m=+702.015196570" Jun 08 18:27:08.268214 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.268142 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj"] Jun 08 18:27:08.271596 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.271572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:08.274075 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.274055 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sknbj\"" Jun 08 18:27:08.289287 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.289254 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj"] Jun 08 18:27:08.313176 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.313153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsdnf\" (UniqueName: \"kubernetes.io/projected/97065893-2e62-497e-94c5-9d5f0058b4f5-kube-api-access-wsdnf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" (UID: \"97065893-2e62-497e-94c5-9d5f0058b4f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:08.313289 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.313220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97065893-2e62-497e-94c5-9d5f0058b4f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" (UID: \"97065893-2e62-497e-94c5-9d5f0058b4f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:08.413980 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.413956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsdnf\" (UniqueName: \"kubernetes.io/projected/97065893-2e62-497e-94c5-9d5f0058b4f5-kube-api-access-wsdnf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" (UID: \"97065893-2e62-497e-94c5-9d5f0058b4f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:08.414084 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.414000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97065893-2e62-497e-94c5-9d5f0058b4f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" (UID: \"97065893-2e62-497e-94c5-9d5f0058b4f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:08.414328 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.414313 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97065893-2e62-497e-94c5-9d5f0058b4f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" (UID: \"97065893-2e62-497e-94c5-9d5f0058b4f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:08.423584 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.423559 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsdnf\" (UniqueName: \"kubernetes.io/projected/97065893-2e62-497e-94c5-9d5f0058b4f5-kube-api-access-wsdnf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" (UID: \"97065893-2e62-497e-94c5-9d5f0058b4f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:08.581660 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.581634 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:08.723038 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.723004 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj"] Jun 08 18:27:08.726476 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:27:08.726446 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97065893_2e62_497e_94c5_9d5f0058b4f5.slice/crio-15708b9bf2ebb5df751642c72e9dafeb578758088c716c762f4cef5f9cb030dc WatchSource:0}: Error finding container 15708b9bf2ebb5df751642c72e9dafeb578758088c716c762f4cef5f9cb030dc: Status 404 returned error can't find the container with id 15708b9bf2ebb5df751642c72e9dafeb578758088c716c762f4cef5f9cb030dc Jun 08 18:27:08.832944 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:08.832879 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" event={"ID":"97065893-2e62-497e-94c5-9d5f0058b4f5","Type":"ContainerStarted","Data":"15708b9bf2ebb5df751642c72e9dafeb578758088c716c762f4cef5f9cb030dc"} Jun 08 18:27:14.864441 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:14.864404 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" event={"ID":"97065893-2e62-497e-94c5-9d5f0058b4f5","Type":"ContainerStarted","Data":"68abd14152ca4c8a3c6756d030776802217b2d20db29e141128152eada45dc82"} Jun 08 18:27:14.864822 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:14.864584 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:14.886469 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:14.886406 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" podStartSLOduration=1.6863161880000002 podStartE2EDuration="6.886388866s" totalImagesPullingTime="5.200072678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:27:14.885984195 +0000 UTC m=+709.046820878" watchObservedRunningTime="2026-06-08 18:27:14.886388866 +0000 UTC m=+709.047225525" Jun 08 18:27:15.817620 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:15.817595 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wqpb5" Jun 08 18:27:18.835274 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:18.835247 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fcm9c" Jun 08 18:27:25.871991 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:25.871964 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:27.141449 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.141414 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z"] Jun 08 18:27:27.146101 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.146083 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.175006 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.174983 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z"] Jun 08 18:27:27.258890 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.258864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ded3c4ba-bb44-4144-aa35-075bd065607c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" (UID: \"ded3c4ba-bb44-4144-aa35-075bd065607c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.259002 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.258927 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tffgz\" (UniqueName: \"kubernetes.io/projected/ded3c4ba-bb44-4144-aa35-075bd065607c-kube-api-access-tffgz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" (UID: \"ded3c4ba-bb44-4144-aa35-075bd065607c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.340820 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.340795 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z"] Jun 08 18:27:27.341037 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:27.341019 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-tffgz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" podUID="ded3c4ba-bb44-4144-aa35-075bd065607c" Jun 08 18:27:27.359568 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.359540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tffgz\" (UniqueName: \"kubernetes.io/projected/ded3c4ba-bb44-4144-aa35-075bd065607c-kube-api-access-tffgz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" (UID: \"ded3c4ba-bb44-4144-aa35-075bd065607c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.359647 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.359588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ded3c4ba-bb44-4144-aa35-075bd065607c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" (UID: \"ded3c4ba-bb44-4144-aa35-075bd065607c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.359932 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.359916 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ded3c4ba-bb44-4144-aa35-075bd065607c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" (UID: \"ded3c4ba-bb44-4144-aa35-075bd065607c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.376912 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.376893 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tffgz\" (UniqueName: \"kubernetes.io/projected/ded3c4ba-bb44-4144-aa35-075bd065607c-kube-api-access-tffgz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" (UID: \"ded3c4ba-bb44-4144-aa35-075bd065607c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.771569 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.771538 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z"] Jun 08 18:27:27.787299 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.787267 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z"] Jun 08 18:27:27.799810 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.799785 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj"] Jun 08 18:27:27.800039 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.799985 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" containerName="manager" containerID="cri-o://68abd14152ca4c8a3c6756d030776802217b2d20db29e141128152eada45dc82" gracePeriod=2 Jun 08 18:27:27.818197 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.818174 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj"] Jun 08 18:27:27.833010 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:27.832979 2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"97065893-2e62-497e-94c5-9d5f0058b4f5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\": pods \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:27.835659 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.835640 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8"] Jun 08 18:27:27.836132 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.836113 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" containerName="manager" Jun 08 18:27:27.836202 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.836134 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" containerName="manager" Jun 08 18:27:27.836202 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.836189 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" containerName="manager" Jun 08 18:27:27.840836 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.840808 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:27.859615 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.859589 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8"] Jun 08 18:27:27.863564 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.863541 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgwwz\" (UniqueName: \"kubernetes.io/projected/a0240118-a8de-4878-b60d-3bed8288cece-kube-api-access-pgwwz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-22bm8\" (UID: \"a0240118-a8de-4878-b60d-3bed8288cece\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:27.863650 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.863579 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0240118-a8de-4878-b60d-3bed8288cece-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-22bm8\" (UID: \"a0240118-a8de-4878-b60d-3bed8288cece\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:27.869184 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.869162 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld"] Jun 08 18:27:27.872465 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.872450 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" Jun 08 18:27:27.875279 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.875258 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qklxh\"" Jun 08 18:27:27.881078 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:27.881049 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:27.890927 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.890906 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld"] Jun 08 18:27:27.922945 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.922921 2560 generic.go:356] "Generic (PLEG): container finished" podID="97065893-2e62-497e-94c5-9d5f0058b4f5" containerID="68abd14152ca4c8a3c6756d030776802217b2d20db29e141128152eada45dc82" exitCode=0 Jun 08 18:27:27.923042 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.923030 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.924665 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:27.924643 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:27.927079 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:27.927054 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="ded3c4ba-bb44-4144-aa35-075bd065607c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.929464 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.929443 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.930823 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:27.930800 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:27.933553 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:27.933526 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="ded3c4ba-bb44-4144-aa35-075bd065607c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:27.937438 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:27.937418 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:27.964922 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.964898 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ded3c4ba-bb44-4144-aa35-075bd065607c-extensions-socket-volume\") pod \"ded3c4ba-bb44-4144-aa35-075bd065607c\" (UID: \"ded3c4ba-bb44-4144-aa35-075bd065607c\") " Jun 08 18:27:27.965018 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.964981 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tffgz\" (UniqueName: \"kubernetes.io/projected/ded3c4ba-bb44-4144-aa35-075bd065607c-kube-api-access-tffgz\") pod \"ded3c4ba-bb44-4144-aa35-075bd065607c\" (UID: \"ded3c4ba-bb44-4144-aa35-075bd065607c\") " Jun 08 18:27:27.965123 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.965106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mb5l\" (UniqueName: \"kubernetes.io/projected/c8eeaa7a-6c43-419b-bccb-f846603904fa-kube-api-access-7mb5l\") pod \"limitador-operator-controller-manager-859d7b78bd-2nwld\" (UID: \"c8eeaa7a-6c43-419b-bccb-f846603904fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" Jun 08 18:27:27.965204 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.965174 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgwwz\" (UniqueName: \"kubernetes.io/projected/a0240118-a8de-4878-b60d-3bed8288cece-kube-api-access-pgwwz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-22bm8\" (UID: \"a0240118-a8de-4878-b60d-3bed8288cece\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:27.965204 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.965183 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded3c4ba-bb44-4144-aa35-075bd065607c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ded3c4ba-bb44-4144-aa35-075bd065607c" (UID: "ded3c4ba-bb44-4144-aa35-075bd065607c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:27:27.965364 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.965222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0240118-a8de-4878-b60d-3bed8288cece-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-22bm8\" (UID: \"a0240118-a8de-4878-b60d-3bed8288cece\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:27.965364 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.965290 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ded3c4ba-bb44-4144-aa35-075bd065607c-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:27:27.965586 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.965570 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0240118-a8de-4878-b60d-3bed8288cece-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-22bm8\" (UID: \"a0240118-a8de-4878-b60d-3bed8288cece\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:27.967264 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.967236 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded3c4ba-bb44-4144-aa35-075bd065607c-kube-api-access-tffgz" (OuterVolumeSpecName: "kube-api-access-tffgz") pod "ded3c4ba-bb44-4144-aa35-075bd065607c" (UID: "ded3c4ba-bb44-4144-aa35-075bd065607c"). InnerVolumeSpecName "kube-api-access-tffgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:27:27.974932 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:27.974893 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgwwz\" (UniqueName: \"kubernetes.io/projected/a0240118-a8de-4878-b60d-3bed8288cece-kube-api-access-pgwwz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-22bm8\" (UID: \"a0240118-a8de-4878-b60d-3bed8288cece\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:28.032479 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.032459 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:28.035090 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:28.035068 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rqqbj\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:28.037280 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:28.037260 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="ded3c4ba-bb44-4144-aa35-075bd065607c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:28.065496 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.065479 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97065893-2e62-497e-94c5-9d5f0058b4f5-extensions-socket-volume\") pod \"97065893-2e62-497e-94c5-9d5f0058b4f5\" (UID: \"97065893-2e62-497e-94c5-9d5f0058b4f5\") " Jun 08 18:27:28.065590 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.065528 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsdnf\" (UniqueName: \"kubernetes.io/projected/97065893-2e62-497e-94c5-9d5f0058b4f5-kube-api-access-wsdnf\") pod \"97065893-2e62-497e-94c5-9d5f0058b4f5\" (UID: \"97065893-2e62-497e-94c5-9d5f0058b4f5\") " Jun 08 18:27:28.065764 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.065747 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mb5l\" (UniqueName: \"kubernetes.io/projected/c8eeaa7a-6c43-419b-bccb-f846603904fa-kube-api-access-7mb5l\") pod \"limitador-operator-controller-manager-859d7b78bd-2nwld\" (UID: \"c8eeaa7a-6c43-419b-bccb-f846603904fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" Jun 08 18:27:28.065817 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.065766 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97065893-2e62-497e-94c5-9d5f0058b4f5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "97065893-2e62-497e-94c5-9d5f0058b4f5" (UID: "97065893-2e62-497e-94c5-9d5f0058b4f5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:27:28.065893 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.065879 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97065893-2e62-497e-94c5-9d5f0058b4f5-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:27:28.065943 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.065896 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tffgz\" (UniqueName: \"kubernetes.io/projected/ded3c4ba-bb44-4144-aa35-075bd065607c-kube-api-access-tffgz\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:27:28.067625 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.067607 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97065893-2e62-497e-94c5-9d5f0058b4f5-kube-api-access-wsdnf" (OuterVolumeSpecName: "kube-api-access-wsdnf") pod "97065893-2e62-497e-94c5-9d5f0058b4f5" (UID: "97065893-2e62-497e-94c5-9d5f0058b4f5"). InnerVolumeSpecName "kube-api-access-wsdnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:27:28.075301 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.075279 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mb5l\" (UniqueName: \"kubernetes.io/projected/c8eeaa7a-6c43-419b-bccb-f846603904fa-kube-api-access-7mb5l\") pod \"limitador-operator-controller-manager-859d7b78bd-2nwld\" (UID: \"c8eeaa7a-6c43-419b-bccb-f846603904fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" Jun 08 18:27:28.166753 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.166726 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsdnf\" (UniqueName: \"kubernetes.io/projected/97065893-2e62-497e-94c5-9d5f0058b4f5-kube-api-access-wsdnf\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:27:28.189822 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.189798 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:28.197158 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.197139 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" Jun 08 18:27:28.337660 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:27:28.337631 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0240118_a8de_4878_b60d_3bed8288cece.slice/crio-103a5d986554ff40b237762e8eb712c5a3b6c17bbb7cd665f2864b00f7460b48 WatchSource:0}: Error finding container 103a5d986554ff40b237762e8eb712c5a3b6c17bbb7cd665f2864b00f7460b48: Status 404 returned error can't find the container with id 103a5d986554ff40b237762e8eb712c5a3b6c17bbb7cd665f2864b00f7460b48 Jun 08 18:27:28.340043 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.340009 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97065893-2e62-497e-94c5-9d5f0058b4f5" path="/var/lib/kubelet/pods/97065893-2e62-497e-94c5-9d5f0058b4f5/volumes" Jun 08 18:27:28.340432 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.340413 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ded3c4ba-bb44-4144-aa35-075bd065607c" path="/var/lib/kubelet/pods/ded3c4ba-bb44-4144-aa35-075bd065607c/volumes" Jun 08 18:27:28.340707 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.340693 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8"] Jun 08 18:27:28.356010 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.355977 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld"] Jun 08 18:27:28.357776 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:27:28.357756 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8eeaa7a_6c43_419b_bccb_f846603904fa.slice/crio-9bbb6f3c86bf06800b3992601fe95ec8205ebfff3c86b20f0dece1c0d60d16fd WatchSource:0}: Error finding container 9bbb6f3c86bf06800b3992601fe95ec8205ebfff3c86b20f0dece1c0d60d16fd: Status 404 returned error can't find the container with id 9bbb6f3c86bf06800b3992601fe95ec8205ebfff3c86b20f0dece1c0d60d16fd Jun 08 18:27:28.930472 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.930435 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" event={"ID":"a0240118-a8de-4878-b60d-3bed8288cece","Type":"ContainerStarted","Data":"6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9"} Jun 08 18:27:28.930647 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.930496 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" event={"ID":"a0240118-a8de-4878-b60d-3bed8288cece","Type":"ContainerStarted","Data":"103a5d986554ff40b237762e8eb712c5a3b6c17bbb7cd665f2864b00f7460b48"} Jun 08 18:27:28.930647 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.930634 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:28.931826 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.931798 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" event={"ID":"c8eeaa7a-6c43-419b-bccb-f846603904fa","Type":"ContainerStarted","Data":"9bbb6f3c86bf06800b3992601fe95ec8205ebfff3c86b20f0dece1c0d60d16fd"} Jun 08 18:27:28.933409 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.933265 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:28.933409 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.933268 2560 scope.go:117] "RemoveContainer" containerID="68abd14152ca4c8a3c6756d030776802217b2d20db29e141128152eada45dc82" Jun 08 18:27:28.933563 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.933271 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rqqbj" Jun 08 18:27:28.954633 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:28.954591 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" podStartSLOduration=1.954577196 podStartE2EDuration="1.954577196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:27:28.953403748 +0000 UTC m=+723.114240408" watchObservedRunningTime="2026-06-08 18:27:28.954577196 +0000 UTC m=+723.115413854" Jun 08 18:27:28.955792 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:28.955759 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="ded3c4ba-bb44-4144-aa35-075bd065607c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:29.939478 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:29.939443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" event={"ID":"c8eeaa7a-6c43-419b-bccb-f846603904fa","Type":"ContainerStarted","Data":"19074f16482a2af173c6978b2e4348ce45214faa603f165804b870ad23a4ca41"} Jun 08 18:27:29.939907 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:29.939569 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" Jun 08 18:27:29.961554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:29.961516 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" podStartSLOduration=1.8372033540000001 podStartE2EDuration="2.961505323s" totalImagesPullingTime="1.124301969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:27:29.959897945 +0000 UTC m=+724.120734600" watchObservedRunningTime="2026-06-08 18:27:29.961505323 +0000 UTC m=+724.122341981" Jun 08 18:27:29.961983 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:29.961958 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8fg4z\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="ded3c4ba-bb44-4144-aa35-075bd065607c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8fg4z" Jun 08 18:27:39.943161 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:39.943132 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:40.952732 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:40.952704 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2nwld" Jun 08 18:27:44.449696 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.449664 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8"] Jun 08 18:27:44.450141 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.449911 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" podUID="a0240118-a8de-4878-b60d-3bed8288cece" containerName="manager" containerID="cri-o://6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9" gracePeriod=10 Jun 08 18:27:44.686932 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.686898 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr"] Jun 08 18:27:44.689406 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.689384 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:44.704676 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.704621 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr"] Jun 08 18:27:44.719213 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.719194 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:44.803641 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.803608 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0240118-a8de-4878-b60d-3bed8288cece-extensions-socket-volume\") pod \"a0240118-a8de-4878-b60d-3bed8288cece\" (UID: \"a0240118-a8de-4878-b60d-3bed8288cece\") " Jun 08 18:27:44.803792 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.803687 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgwwz\" (UniqueName: \"kubernetes.io/projected/a0240118-a8de-4878-b60d-3bed8288cece-kube-api-access-pgwwz\") pod \"a0240118-a8de-4878-b60d-3bed8288cece\" (UID: \"a0240118-a8de-4878-b60d-3bed8288cece\") " Jun 08 18:27:44.803858 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.803826 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3109d910-1b1c-4a16-b55c-ab694f133abe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-464nr\" (UID: \"3109d910-1b1c-4a16-b55c-ab694f133abe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:44.803978 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.803956 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0240118-a8de-4878-b60d-3bed8288cece-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a0240118-a8de-4878-b60d-3bed8288cece" (UID: "a0240118-a8de-4878-b60d-3bed8288cece"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:27:44.804015 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.803973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb9dm\" (UniqueName: \"kubernetes.io/projected/3109d910-1b1c-4a16-b55c-ab694f133abe-kube-api-access-qb9dm\") pod \"kuadrant-operator-controller-manager-86878c769f-464nr\" (UID: \"3109d910-1b1c-4a16-b55c-ab694f133abe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:44.804049 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.804019 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0240118-a8de-4878-b60d-3bed8288cece-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:27:44.805939 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.805913 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0240118-a8de-4878-b60d-3bed8288cece-kube-api-access-pgwwz" (OuterVolumeSpecName: "kube-api-access-pgwwz") pod "a0240118-a8de-4878-b60d-3bed8288cece" (UID: "a0240118-a8de-4878-b60d-3bed8288cece"). InnerVolumeSpecName "kube-api-access-pgwwz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:27:44.904459 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.904429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3109d910-1b1c-4a16-b55c-ab694f133abe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-464nr\" (UID: \"3109d910-1b1c-4a16-b55c-ab694f133abe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:44.904596 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.904506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb9dm\" (UniqueName: \"kubernetes.io/projected/3109d910-1b1c-4a16-b55c-ab694f133abe-kube-api-access-qb9dm\") pod \"kuadrant-operator-controller-manager-86878c769f-464nr\" (UID: \"3109d910-1b1c-4a16-b55c-ab694f133abe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:44.904596 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.904545 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgwwz\" (UniqueName: \"kubernetes.io/projected/a0240118-a8de-4878-b60d-3bed8288cece-kube-api-access-pgwwz\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:27:44.904777 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.904757 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3109d910-1b1c-4a16-b55c-ab694f133abe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-464nr\" (UID: \"3109d910-1b1c-4a16-b55c-ab694f133abe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:44.916184 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:44.916163 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb9dm\" (UniqueName: \"kubernetes.io/projected/3109d910-1b1c-4a16-b55c-ab694f133abe-kube-api-access-qb9dm\") pod \"kuadrant-operator-controller-manager-86878c769f-464nr\" (UID: \"3109d910-1b1c-4a16-b55c-ab694f133abe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:45.014436 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.014365 2560 generic.go:356] "Generic (PLEG): container finished" podID="a0240118-a8de-4878-b60d-3bed8288cece" containerID="6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9" exitCode=0 Jun 08 18:27:45.014436 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.014415 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" Jun 08 18:27:45.014595 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.014446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" event={"ID":"a0240118-a8de-4878-b60d-3bed8288cece","Type":"ContainerDied","Data":"6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9"} Jun 08 18:27:45.014595 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.014503 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8" event={"ID":"a0240118-a8de-4878-b60d-3bed8288cece","Type":"ContainerDied","Data":"103a5d986554ff40b237762e8eb712c5a3b6c17bbb7cd665f2864b00f7460b48"} Jun 08 18:27:45.014595 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.014532 2560 scope.go:117] "RemoveContainer" containerID="6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9" Jun 08 18:27:45.015464 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.015446 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:45.027014 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.026993 2560 scope.go:117] "RemoveContainer" containerID="6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9" Jun 08 18:27:45.027286 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:27:45.027266 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9\": container with ID starting with 6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9 not found: ID does not exist" containerID="6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9" Jun 08 18:27:45.027348 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.027297 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9"} err="failed to get container status \"6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9\": rpc error: code = NotFound desc = could not find container \"6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9\": container with ID starting with 6a7f2eb436dc81e81be1f6532b3a7ee97f469fd44372fe21d600746e8339fcb9 not found: ID does not exist" Jun 08 18:27:45.037929 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.037907 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8"] Jun 08 18:27:45.044236 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.044214 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-22bm8"] Jun 08 18:27:45.159475 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:45.159452 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr"] Jun 08 18:27:45.164378 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:27:45.162974 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3109d910_1b1c_4a16_b55c_ab694f133abe.slice/crio-fe11e3034c022544ee10509c0c5b8a049f2c10668d714b2af52ced0395657649 WatchSource:0}: Error finding container fe11e3034c022544ee10509c0c5b8a049f2c10668d714b2af52ced0395657649: Status 404 returned error can't find the container with id fe11e3034c022544ee10509c0c5b8a049f2c10668d714b2af52ced0395657649 Jun 08 18:27:46.020523 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:46.020492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" event={"ID":"3109d910-1b1c-4a16-b55c-ab694f133abe","Type":"ContainerStarted","Data":"d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62"} Jun 08 18:27:46.020958 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:46.020550 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:27:46.020958 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:46.020569 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" event={"ID":"3109d910-1b1c-4a16-b55c-ab694f133abe","Type":"ContainerStarted","Data":"fe11e3034c022544ee10509c0c5b8a049f2c10668d714b2af52ced0395657649"} Jun 08 18:27:46.044416 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:46.044373 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" podStartSLOduration=2.044358956 podStartE2EDuration="2.044358956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:27:46.041704698 +0000 UTC m=+740.202541356" watchObservedRunningTime="2026-06-08 18:27:46.044358956 +0000 UTC m=+740.205195645" Jun 08 18:27:46.337519 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:46.337485 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0240118-a8de-4878-b60d-3bed8288cece" path="/var/lib/kubelet/pods/a0240118-a8de-4878-b60d-3bed8288cece/volumes" Jun 08 18:27:57.028573 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:27:57.028542 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:28:16.733777 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.733740 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-ldv75"] Jun 08 18:28:16.734366 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.734122 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0240118-a8de-4878-b60d-3bed8288cece" containerName="manager" Jun 08 18:28:16.734366 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.734138 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0240118-a8de-4878-b60d-3bed8288cece" containerName="manager" Jun 08 18:28:16.734366 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.734221 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0240118-a8de-4878-b60d-3bed8288cece" containerName="manager" Jun 08 18:28:16.737106 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.737090 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ldv75" Jun 08 18:28:16.739648 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.739628 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xq7r4\"" Jun 08 18:28:16.743070 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.743042 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ldv75"] Jun 08 18:28:16.775807 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.775781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctsh6\" (UniqueName: \"kubernetes.io/projected/c5059c3a-1f52-4554-ac57-51701dbdeda5-kube-api-access-ctsh6\") pod \"authorino-75d556bd77-ldv75\" (UID: \"c5059c3a-1f52-4554-ac57-51701dbdeda5\") " pod="kuadrant-system/authorino-75d556bd77-ldv75" Jun 08 18:28:16.877051 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.877010 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctsh6\" (UniqueName: \"kubernetes.io/projected/c5059c3a-1f52-4554-ac57-51701dbdeda5-kube-api-access-ctsh6\") pod \"authorino-75d556bd77-ldv75\" (UID: \"c5059c3a-1f52-4554-ac57-51701dbdeda5\") " pod="kuadrant-system/authorino-75d556bd77-ldv75" Jun 08 18:28:16.885681 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:16.885661 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctsh6\" (UniqueName: \"kubernetes.io/projected/c5059c3a-1f52-4554-ac57-51701dbdeda5-kube-api-access-ctsh6\") pod \"authorino-75d556bd77-ldv75\" (UID: \"c5059c3a-1f52-4554-ac57-51701dbdeda5\") " pod="kuadrant-system/authorino-75d556bd77-ldv75" Jun 08 18:28:17.048041 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:17.047965 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ldv75" Jun 08 18:28:17.218083 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:17.217987 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ldv75"] Jun 08 18:28:17.220262 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:28:17.220230 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5059c3a_1f52_4554_ac57_51701dbdeda5.slice/crio-2d1e3e028e9be41b5209c0fa90b99fc6ab0c2ec0b3c47ee930a98dc9b26d197c WatchSource:0}: Error finding container 2d1e3e028e9be41b5209c0fa90b99fc6ab0c2ec0b3c47ee930a98dc9b26d197c: Status 404 returned error can't find the container with id 2d1e3e028e9be41b5209c0fa90b99fc6ab0c2ec0b3c47ee930a98dc9b26d197c Jun 08 18:28:18.182114 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:18.182071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ldv75" event={"ID":"c5059c3a-1f52-4554-ac57-51701dbdeda5","Type":"ContainerStarted","Data":"2d1e3e028e9be41b5209c0fa90b99fc6ab0c2ec0b3c47ee930a98dc9b26d197c"} Jun 08 18:28:20.193388 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:20.193351 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ldv75" event={"ID":"c5059c3a-1f52-4554-ac57-51701dbdeda5","Type":"ContainerStarted","Data":"15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0"} Jun 08 18:28:20.222559 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:28:20.222507 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-ldv75" podStartSLOduration=1.4227099619999999 podStartE2EDuration="4.222492691s" totalImagesPullingTime="2.799782729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:28:20.221548351 +0000 UTC m=+774.382385031" watchObservedRunningTime="2026-06-08 18:28:20.222492691 +0000 UTC m=+774.383329348" Jun 08 18:29:11.164423 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.164343 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:29:11.167779 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.167760 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:11.170410 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.170384 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 08 18:29:11.170537 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.170448 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 08 18:29:11.171459 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.171446 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 08 18:29:11.171459 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.171453 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-qq2bh\"" Jun 08 18:29:11.177556 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.177538 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:29:11.238491 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.238464 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29k49\" (UniqueName: \"kubernetes.io/projected/f1e5624d-e0e7-4635-a95f-0d31d35e5704-kube-api-access-29k49\") pod \"maas-keycloak-0\" (UID: \"f1e5624d-e0e7-4635-a95f-0d31d35e5704\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:11.339158 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.339129 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29k49\" (UniqueName: \"kubernetes.io/projected/f1e5624d-e0e7-4635-a95f-0d31d35e5704-kube-api-access-29k49\") pod \"maas-keycloak-0\" (UID: \"f1e5624d-e0e7-4635-a95f-0d31d35e5704\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:11.347411 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.347385 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29k49\" (UniqueName: \"kubernetes.io/projected/f1e5624d-e0e7-4635-a95f-0d31d35e5704-kube-api-access-29k49\") pod \"maas-keycloak-0\" (UID: \"f1e5624d-e0e7-4635-a95f-0d31d35e5704\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:11.477656 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.477581 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:11.610203 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:11.610180 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:29:11.612406 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:29:11.612383 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e5624d_e0e7_4635_a95f_0d31d35e5704.slice/crio-95cc909ad9d0576e27994473ff81acb6a48a4c8507dcfa80a047a4de1d3b884c WatchSource:0}: Error finding container 95cc909ad9d0576e27994473ff81acb6a48a4c8507dcfa80a047a4de1d3b884c: Status 404 returned error can't find the container with id 95cc909ad9d0576e27994473ff81acb6a48a4c8507dcfa80a047a4de1d3b884c Jun 08 18:29:12.434276 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:12.434237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f1e5624d-e0e7-4635-a95f-0d31d35e5704","Type":"ContainerStarted","Data":"95cc909ad9d0576e27994473ff81acb6a48a4c8507dcfa80a047a4de1d3b884c"} Jun 08 18:29:16.458385 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:16.458288 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f1e5624d-e0e7-4635-a95f-0d31d35e5704","Type":"ContainerStarted","Data":"3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e"} Jun 08 18:29:16.477792 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:16.477756 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:16.479524 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:16.479488 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:16.489413 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:16.489342 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.904745616 podStartE2EDuration="5.489323523s" totalImagesPullingTime="4.584577907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:29:16.485735313 +0000 UTC m=+830.646571971" watchObservedRunningTime="2026-06-08 18:29:16.489323523 +0000 UTC m=+830.650160182" Jun 08 18:29:17.478717 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:17.478670 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:18.478942 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:18.478891 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:19.478601 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:19.478549 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:20.480353 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:20.480302 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:21.478618 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:21.478578 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:21.480394 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:21.479229 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:22.478478 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:22.478426 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:23.478541 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:23.478492 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:24.478657 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:24.478617 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:25.478240 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:25.478185 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:26.478055 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:26.478011 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:27.478365 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:27.478310 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:28.477962 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:28.477920 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 08 18:29:29.592617 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:29.592580 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:29.610973 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:29.610924 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:29:39.599923 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:39.599878 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 08 18:29:41.933773 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:41.933736 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ldv75"] Jun 08 18:29:41.934257 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:41.933999 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-ldv75" podUID="c5059c3a-1f52-4554-ac57-51701dbdeda5" containerName="authorino" containerID="cri-o://15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0" gracePeriod=30 Jun 08 18:29:42.190936 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.190882 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ldv75" Jun 08 18:29:42.341791 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.341762 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctsh6\" (UniqueName: \"kubernetes.io/projected/c5059c3a-1f52-4554-ac57-51701dbdeda5-kube-api-access-ctsh6\") pod \"c5059c3a-1f52-4554-ac57-51701dbdeda5\" (UID: \"c5059c3a-1f52-4554-ac57-51701dbdeda5\") " Jun 08 18:29:42.344114 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.344082 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5059c3a-1f52-4554-ac57-51701dbdeda5-kube-api-access-ctsh6" (OuterVolumeSpecName: "kube-api-access-ctsh6") pod "c5059c3a-1f52-4554-ac57-51701dbdeda5" (UID: "c5059c3a-1f52-4554-ac57-51701dbdeda5"). InnerVolumeSpecName "kube-api-access-ctsh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:29:42.443104 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.443024 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctsh6\" (UniqueName: \"kubernetes.io/projected/c5059c3a-1f52-4554-ac57-51701dbdeda5-kube-api-access-ctsh6\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:29:42.621593 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.621560 2560 generic.go:356] "Generic (PLEG): container finished" podID="c5059c3a-1f52-4554-ac57-51701dbdeda5" containerID="15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0" exitCode=0 Jun 08 18:29:42.621593 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.621599 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ldv75" Jun 08 18:29:42.621807 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.621645 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ldv75" event={"ID":"c5059c3a-1f52-4554-ac57-51701dbdeda5","Type":"ContainerDied","Data":"15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0"} Jun 08 18:29:42.621807 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.621685 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ldv75" event={"ID":"c5059c3a-1f52-4554-ac57-51701dbdeda5","Type":"ContainerDied","Data":"2d1e3e028e9be41b5209c0fa90b99fc6ab0c2ec0b3c47ee930a98dc9b26d197c"} Jun 08 18:29:42.621807 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.621703 2560 scope.go:117] "RemoveContainer" containerID="15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0" Jun 08 18:29:42.630330 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.630310 2560 scope.go:117] "RemoveContainer" containerID="15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0" Jun 08 18:29:42.630607 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:29:42.630587 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0\": container with ID starting with 15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0 not found: ID does not exist" containerID="15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0" Jun 08 18:29:42.630652 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.630617 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0"} err="failed to get container status \"15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0\": rpc error: code = NotFound desc = could not find container \"15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0\": container with ID starting with 15e324b2416d4a072caf1094b8266677a024dc5dbec052110127f53ad93851b0 not found: ID does not exist" Jun 08 18:29:42.642575 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.642549 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ldv75"] Jun 08 18:29:42.646458 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:42.646438 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ldv75"] Jun 08 18:29:44.337584 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:44.337551 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5059c3a-1f52-4554-ac57-51701dbdeda5" path="/var/lib/kubelet/pods/c5059c3a-1f52-4554-ac57-51701dbdeda5/volumes" Jun 08 18:29:45.425756 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.425719 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f96d9445f-7tm7p"] Jun 08 18:29:45.426275 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.426258 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5059c3a-1f52-4554-ac57-51701dbdeda5" containerName="authorino" Jun 08 18:29:45.426339 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.426278 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5059c3a-1f52-4554-ac57-51701dbdeda5" containerName="authorino" Jun 08 18:29:45.426377 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.426362 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5059c3a-1f52-4554-ac57-51701dbdeda5" containerName="authorino" Jun 08 18:29:45.429367 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.429347 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:45.435172 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.432168 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 18:29:45.435172 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.432669 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 18:29:45.435172 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.433179 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-25l86\"" Jun 08 18:29:45.451877 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.451835 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f96d9445f-7tm7p"] Jun 08 18:29:45.470561 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.470535 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hmqv\" (UniqueName: \"kubernetes.io/projected/b0f8f70a-b890-4a92-8440-d91aba00b4dd-kube-api-access-7hmqv\") pod \"maas-controller-5f96d9445f-7tm7p\" (UID: \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\") " pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:45.470679 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.470572 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0f8f70a-b890-4a92-8440-d91aba00b4dd-webhook-cert\") pod \"maas-controller-5f96d9445f-7tm7p\" (UID: \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\") " pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:45.571931 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.571896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hmqv\" (UniqueName: \"kubernetes.io/projected/b0f8f70a-b890-4a92-8440-d91aba00b4dd-kube-api-access-7hmqv\") pod \"maas-controller-5f96d9445f-7tm7p\" (UID: \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\") " pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:45.572094 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.571936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0f8f70a-b890-4a92-8440-d91aba00b4dd-webhook-cert\") pod \"maas-controller-5f96d9445f-7tm7p\" (UID: \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\") " pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:45.574594 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.574561 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0f8f70a-b890-4a92-8440-d91aba00b4dd-webhook-cert\") pod \"maas-controller-5f96d9445f-7tm7p\" (UID: \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\") " pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:45.582663 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.582578 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hmqv\" (UniqueName: \"kubernetes.io/projected/b0f8f70a-b890-4a92-8440-d91aba00b4dd-kube-api-access-7hmqv\") pod \"maas-controller-5f96d9445f-7tm7p\" (UID: \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\") " pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:45.591912 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.591889 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cb9f9847-hd869"] Jun 08 18:29:45.594228 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.594212 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:45.606595 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.606574 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb9f9847-hd869"] Jun 08 18:29:45.673047 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.673013 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d42a19e-882c-4c11-86c9-c3e72a380c83-webhook-cert\") pod \"maas-controller-6cb9f9847-hd869\" (UID: \"2d42a19e-882c-4c11-86c9-c3e72a380c83\") " pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:45.673202 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.673072 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vttf4\" (UniqueName: \"kubernetes.io/projected/2d42a19e-882c-4c11-86c9-c3e72a380c83-kube-api-access-vttf4\") pod \"maas-controller-6cb9f9847-hd869\" (UID: \"2d42a19e-882c-4c11-86c9-c3e72a380c83\") " pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:45.746007 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.745945 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:45.773758 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.773731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d42a19e-882c-4c11-86c9-c3e72a380c83-webhook-cert\") pod \"maas-controller-6cb9f9847-hd869\" (UID: \"2d42a19e-882c-4c11-86c9-c3e72a380c83\") " pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:45.773881 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.773788 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vttf4\" (UniqueName: \"kubernetes.io/projected/2d42a19e-882c-4c11-86c9-c3e72a380c83-kube-api-access-vttf4\") pod \"maas-controller-6cb9f9847-hd869\" (UID: \"2d42a19e-882c-4c11-86c9-c3e72a380c83\") " pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:45.776592 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.776568 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d42a19e-882c-4c11-86c9-c3e72a380c83-webhook-cert\") pod \"maas-controller-6cb9f9847-hd869\" (UID: \"2d42a19e-882c-4c11-86c9-c3e72a380c83\") " pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:45.783068 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.783043 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vttf4\" (UniqueName: \"kubernetes.io/projected/2d42a19e-882c-4c11-86c9-c3e72a380c83-kube-api-access-vttf4\") pod \"maas-controller-6cb9f9847-hd869\" (UID: \"2d42a19e-882c-4c11-86c9-c3e72a380c83\") " pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:45.876670 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.876646 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f96d9445f-7tm7p"] Jun 08 18:29:45.878955 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:29:45.878920 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f8f70a_b890_4a92_8440_d91aba00b4dd.slice/crio-9404fd319395924b6500b94fb46ee5d1ffd017e1e8f04db2750effdba83eb3e5 WatchSource:0}: Error finding container 9404fd319395924b6500b94fb46ee5d1ffd017e1e8f04db2750effdba83eb3e5: Status 404 returned error can't find the container with id 9404fd319395924b6500b94fb46ee5d1ffd017e1e8f04db2750effdba83eb3e5 Jun 08 18:29:45.904303 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:45.904275 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:46.033967 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:46.033941 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb9f9847-hd869"] Jun 08 18:29:46.035315 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:29:46.035291 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d42a19e_882c_4c11_86c9_c3e72a380c83.slice/crio-8d1cbd1bf447a2b479ff706fe5952e1256d9cb50a92355bc4bd3f38012959055 WatchSource:0}: Error finding container 8d1cbd1bf447a2b479ff706fe5952e1256d9cb50a92355bc4bd3f38012959055: Status 404 returned error can't find the container with id 8d1cbd1bf447a2b479ff706fe5952e1256d9cb50a92355bc4bd3f38012959055 Jun 08 18:29:46.645335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:46.645293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb9f9847-hd869" event={"ID":"2d42a19e-882c-4c11-86c9-c3e72a380c83","Type":"ContainerStarted","Data":"8d1cbd1bf447a2b479ff706fe5952e1256d9cb50a92355bc4bd3f38012959055"} Jun 08 18:29:46.646810 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:46.646780 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" event={"ID":"b0f8f70a-b890-4a92-8440-d91aba00b4dd","Type":"ContainerStarted","Data":"9404fd319395924b6500b94fb46ee5d1ffd017e1e8f04db2750effdba83eb3e5"} Jun 08 18:29:48.661938 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:48.661899 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" event={"ID":"b0f8f70a-b890-4a92-8440-d91aba00b4dd","Type":"ContainerStarted","Data":"f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e"} Jun 08 18:29:48.662445 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:48.661964 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:48.663451 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:48.663418 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb9f9847-hd869" event={"ID":"2d42a19e-882c-4c11-86c9-c3e72a380c83","Type":"ContainerStarted","Data":"e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732"} Jun 08 18:29:48.663584 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:48.663541 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:48.693169 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:48.693119 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" podStartSLOduration=1.067749847 podStartE2EDuration="3.693101456s" totalImagesPullingTime="2.625351609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:29:48.690533248 +0000 UTC m=+862.851369907" watchObservedRunningTime="2026-06-08 18:29:48.693101456 +0000 UTC m=+862.853938117" Jun 08 18:29:48.716124 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:48.716065 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cb9f9847-hd869" podStartSLOduration=1.253908936 podStartE2EDuration="3.716045501s" totalImagesPullingTime="2.462136565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:29:48.714284903 +0000 UTC m=+862.875121562" watchObservedRunningTime="2026-06-08 18:29:48.716045501 +0000 UTC m=+862.876882160" Jun 08 18:29:59.674249 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:59.674221 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:29:59.674691 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:59.674285 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:29:59.784856 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:59.784807 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f96d9445f-7tm7p"] Jun 08 18:29:59.785033 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:29:59.785012 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" podUID="b0f8f70a-b890-4a92-8440-d91aba00b4dd" containerName="manager" containerID="cri-o://f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e" gracePeriod=30 Jun 08 18:30:00.030079 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.030059 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:30:00.100614 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.100588 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0f8f70a-b890-4a92-8440-d91aba00b4dd-webhook-cert\") pod \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\" (UID: \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\") " Jun 08 18:30:00.100756 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.100645 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hmqv\" (UniqueName: \"kubernetes.io/projected/b0f8f70a-b890-4a92-8440-d91aba00b4dd-kube-api-access-7hmqv\") pod \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\" (UID: \"b0f8f70a-b890-4a92-8440-d91aba00b4dd\") " Jun 08 18:30:00.102931 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.102902 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8f70a-b890-4a92-8440-d91aba00b4dd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b0f8f70a-b890-4a92-8440-d91aba00b4dd" (UID: "b0f8f70a-b890-4a92-8440-d91aba00b4dd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:30:00.103022 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.102969 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f8f70a-b890-4a92-8440-d91aba00b4dd-kube-api-access-7hmqv" (OuterVolumeSpecName: "kube-api-access-7hmqv") pod "b0f8f70a-b890-4a92-8440-d91aba00b4dd" (UID: "b0f8f70a-b890-4a92-8440-d91aba00b4dd"). InnerVolumeSpecName "kube-api-access-7hmqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:30:00.201756 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.201703 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0f8f70a-b890-4a92-8440-d91aba00b4dd-webhook-cert\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:30:00.201756 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.201727 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hmqv\" (UniqueName: \"kubernetes.io/projected/b0f8f70a-b890-4a92-8440-d91aba00b4dd-kube-api-access-7hmqv\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:30:00.719495 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.719448 2560 generic.go:356] "Generic (PLEG): container finished" podID="b0f8f70a-b890-4a92-8440-d91aba00b4dd" containerID="f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e" exitCode=0 Jun 08 18:30:00.719989 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.719501 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" Jun 08 18:30:00.719989 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.719534 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" event={"ID":"b0f8f70a-b890-4a92-8440-d91aba00b4dd","Type":"ContainerDied","Data":"f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e"} Jun 08 18:30:00.719989 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.719589 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f96d9445f-7tm7p" event={"ID":"b0f8f70a-b890-4a92-8440-d91aba00b4dd","Type":"ContainerDied","Data":"9404fd319395924b6500b94fb46ee5d1ffd017e1e8f04db2750effdba83eb3e5"} Jun 08 18:30:00.719989 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.719615 2560 scope.go:117] "RemoveContainer" containerID="f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e" Jun 08 18:30:00.728276 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.728258 2560 scope.go:117] "RemoveContainer" containerID="f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e" Jun 08 18:30:00.728544 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:30:00.728526 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e\": container with ID starting with f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e not found: ID does not exist" containerID="f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e" Jun 08 18:30:00.728597 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.728554 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e"} err="failed to get container status \"f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e\": rpc error: code = NotFound desc = could not find container \"f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e\": container with ID starting with f8aceda622fae6e9a6bfd5740e2a88bdb6025fcbcc441a3c2f643f899c92599e not found: ID does not exist" Jun 08 18:30:00.739369 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.739350 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f96d9445f-7tm7p"] Jun 08 18:30:00.744509 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:00.744483 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f96d9445f-7tm7p"] Jun 08 18:30:02.338328 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:02.338295 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f8f70a-b890-4a92-8440-d91aba00b4dd" path="/var/lib/kubelet/pods/b0f8f70a-b890-4a92-8440-d91aba00b4dd/volumes" Jun 08 18:30:19.753833 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.753743 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-77d5447464-822qr"] Jun 08 18:30:19.754544 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.754134 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f8f70a-b890-4a92-8440-d91aba00b4dd" containerName="manager" Jun 08 18:30:19.754544 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.754148 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8f70a-b890-4a92-8440-d91aba00b4dd" containerName="manager" Jun 08 18:30:19.754544 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.754237 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f8f70a-b890-4a92-8440-d91aba00b4dd" containerName="manager" Jun 08 18:30:19.758745 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.758725 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:19.761783 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.761762 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 08 18:30:19.761783 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.761779 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-vmlgf\"" Jun 08 18:30:19.771671 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.771642 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-77d5447464-822qr"] Jun 08 18:30:19.861784 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.861756 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9bca354-5115-46fb-b6de-3981ba703534-maas-api-tls\") pod \"maas-api-77d5447464-822qr\" (UID: \"f9bca354-5115-46fb-b6de-3981ba703534\") " pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:19.861916 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.861861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk4jv\" (UniqueName: \"kubernetes.io/projected/f9bca354-5115-46fb-b6de-3981ba703534-kube-api-access-qk4jv\") pod \"maas-api-77d5447464-822qr\" (UID: \"f9bca354-5115-46fb-b6de-3981ba703534\") " pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:19.962680 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.962650 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9bca354-5115-46fb-b6de-3981ba703534-maas-api-tls\") pod \"maas-api-77d5447464-822qr\" (UID: \"f9bca354-5115-46fb-b6de-3981ba703534\") " pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:19.962813 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.962749 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk4jv\" (UniqueName: \"kubernetes.io/projected/f9bca354-5115-46fb-b6de-3981ba703534-kube-api-access-qk4jv\") pod \"maas-api-77d5447464-822qr\" (UID: \"f9bca354-5115-46fb-b6de-3981ba703534\") " pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:19.965412 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.965388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9bca354-5115-46fb-b6de-3981ba703534-maas-api-tls\") pod \"maas-api-77d5447464-822qr\" (UID: \"f9bca354-5115-46fb-b6de-3981ba703534\") " pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:19.972747 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:19.972695 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk4jv\" (UniqueName: \"kubernetes.io/projected/f9bca354-5115-46fb-b6de-3981ba703534-kube-api-access-qk4jv\") pod \"maas-api-77d5447464-822qr\" (UID: \"f9bca354-5115-46fb-b6de-3981ba703534\") " pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:20.087668 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.087637 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:20.232607 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.232583 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-77d5447464-822qr"] Jun 08 18:30:20.234755 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:30:20.234732 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9bca354_5115_46fb_b6de_3981ba703534.slice/crio-3d08658256d4fcf53d1188e381a2e2ea97ee8f05c2d0ec8e1749f241fb1e66ec WatchSource:0}: Error finding container 3d08658256d4fcf53d1188e381a2e2ea97ee8f05c2d0ec8e1749f241fb1e66ec: Status 404 returned error can't find the container with id 3d08658256d4fcf53d1188e381a2e2ea97ee8f05c2d0ec8e1749f241fb1e66ec Jun 08 18:30:20.255540 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.255516 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-p5fnj"] Jun 08 18:30:20.258765 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.258748 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" Jun 08 18:30:20.261104 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.261082 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 08 18:30:20.261222 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.261206 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bc7d6\"" Jun 08 18:30:20.268077 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.268059 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-p5fnj"] Jun 08 18:30:20.282517 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.282491 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-9m7hr"] Jun 08 18:30:20.285838 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.285813 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" Jun 08 18:30:20.306920 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.306892 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-9m7hr"] Jun 08 18:30:20.367421 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.367348 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlqg4\" (UniqueName: \"kubernetes.io/projected/74581d47-cce4-49b4-a980-0cc0e48c05ce-kube-api-access-tlqg4\") pod \"payload-processing-7c6c859784-9m7hr\" (UID: \"74581d47-cce4-49b4-a980-0cc0e48c05ce\") " pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" Jun 08 18:30:20.367531 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.367489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfw4t\" (UniqueName: \"kubernetes.io/projected/a23ba43d-8f9b-47f1-a68c-b175c25bdc95-kube-api-access-qfw4t\") pod \"payload-pre-processing-67d7689b65-p5fnj\" (UID: \"a23ba43d-8f9b-47f1-a68c-b175c25bdc95\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" Jun 08 18:30:20.468319 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.468288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlqg4\" (UniqueName: \"kubernetes.io/projected/74581d47-cce4-49b4-a980-0cc0e48c05ce-kube-api-access-tlqg4\") pod \"payload-processing-7c6c859784-9m7hr\" (UID: \"74581d47-cce4-49b4-a980-0cc0e48c05ce\") " pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" Jun 08 18:30:20.468507 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.468438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfw4t\" (UniqueName: \"kubernetes.io/projected/a23ba43d-8f9b-47f1-a68c-b175c25bdc95-kube-api-access-qfw4t\") pod \"payload-pre-processing-67d7689b65-p5fnj\" (UID: \"a23ba43d-8f9b-47f1-a68c-b175c25bdc95\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" Jun 08 18:30:20.483250 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.483221 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfw4t\" (UniqueName: \"kubernetes.io/projected/a23ba43d-8f9b-47f1-a68c-b175c25bdc95-kube-api-access-qfw4t\") pod \"payload-pre-processing-67d7689b65-p5fnj\" (UID: \"a23ba43d-8f9b-47f1-a68c-b175c25bdc95\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" Jun 08 18:30:20.483427 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.483404 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlqg4\" (UniqueName: \"kubernetes.io/projected/74581d47-cce4-49b4-a980-0cc0e48c05ce-kube-api-access-tlqg4\") pod \"payload-processing-7c6c859784-9m7hr\" (UID: \"74581d47-cce4-49b4-a980-0cc0e48c05ce\") " pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" Jun 08 18:30:20.570295 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.570261 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" Jun 08 18:30:20.598303 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.598276 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" Jun 08 18:30:20.725760 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.725723 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-p5fnj"] Jun 08 18:30:20.727455 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:30:20.727430 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23ba43d_8f9b_47f1_a68c_b175c25bdc95.slice/crio-87bd98283f096f18004f575fb6ac66c6c0f1e9697d20daad3a73587db1af9beb WatchSource:0}: Error finding container 87bd98283f096f18004f575fb6ac66c6c0f1e9697d20daad3a73587db1af9beb: Status 404 returned error can't find the container with id 87bd98283f096f18004f575fb6ac66c6c0f1e9697d20daad3a73587db1af9beb Jun 08 18:30:20.745502 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.745484 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-9m7hr"] Jun 08 18:30:20.747569 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:30:20.747536 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74581d47_cce4_49b4_a980_0cc0e48c05ce.slice/crio-34636e5a1b90e5769289993cd21efa7d1f774ad401ffa8e9c13fb1b8ac0f9a29 WatchSource:0}: Error finding container 34636e5a1b90e5769289993cd21efa7d1f774ad401ffa8e9c13fb1b8ac0f9a29: Status 404 returned error can't find the container with id 34636e5a1b90e5769289993cd21efa7d1f774ad401ffa8e9c13fb1b8ac0f9a29 Jun 08 18:30:20.815901 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.815874 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" event={"ID":"74581d47-cce4-49b4-a980-0cc0e48c05ce","Type":"ContainerStarted","Data":"34636e5a1b90e5769289993cd21efa7d1f774ad401ffa8e9c13fb1b8ac0f9a29"} Jun 08 18:30:20.817137 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.817112 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" event={"ID":"a23ba43d-8f9b-47f1-a68c-b175c25bdc95","Type":"ContainerStarted","Data":"87bd98283f096f18004f575fb6ac66c6c0f1e9697d20daad3a73587db1af9beb"} Jun 08 18:30:20.818256 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:20.818238 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-77d5447464-822qr" event={"ID":"f9bca354-5115-46fb-b6de-3981ba703534","Type":"ContainerStarted","Data":"3d08658256d4fcf53d1188e381a2e2ea97ee8f05c2d0ec8e1749f241fb1e66ec"} Jun 08 18:30:22.831088 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.831049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" event={"ID":"74581d47-cce4-49b4-a980-0cc0e48c05ce","Type":"ContainerStarted","Data":"2839a07f8076668c79c9a7af16245bdbe8633fd4e939593f5f898a089914d456"} Jun 08 18:30:22.831503 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.831118 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" Jun 08 18:30:22.832630 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.832603 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" event={"ID":"a23ba43d-8f9b-47f1-a68c-b175c25bdc95","Type":"ContainerStarted","Data":"d19849608b0e8c7210b4485ad0e50845ca10429da7585916e9226c502af1b598"} Jun 08 18:30:22.832754 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.832660 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" Jun 08 18:30:22.834005 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.833983 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-77d5447464-822qr" event={"ID":"f9bca354-5115-46fb-b6de-3981ba703534","Type":"ContainerStarted","Data":"f510e134e4af345c729c79a3042b0a2d583081003b019da4fd08ba2ed87767fa"} Jun 08 18:30:22.834124 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.834111 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:22.856237 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.856195 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" podStartSLOduration=1.315039459 podStartE2EDuration="2.856183761s" totalImagesPullingTime="1.541144302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:30:22.854590267 +0000 UTC m=+897.015426926" watchObservedRunningTime="2026-06-08 18:30:22.856183761 +0000 UTC m=+897.017020418" Jun 08 18:30:22.878350 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.878307 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-77d5447464-822qr" podStartSLOduration=1.826984959 podStartE2EDuration="3.878293868s" totalImagesPullingTime="2.051308909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:30:22.877743482 +0000 UTC m=+897.038580139" watchObservedRunningTime="2026-06-08 18:30:22.878293868 +0000 UTC m=+897.039130522" Jun 08 18:30:22.901751 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:22.901714 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" podStartSLOduration=1.343567092 podStartE2EDuration="2.901698392s" totalImagesPullingTime="1.5581313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:30:22.901240813 +0000 UTC m=+897.062077474" watchObservedRunningTime="2026-06-08 18:30:22.901698392 +0000 UTC m=+897.062535048" Jun 08 18:30:26.354040 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:26.354008 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:30:26.355093 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:26.355071 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:30:28.843226 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:28.843201 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-77d5447464-822qr" Jun 08 18:30:30.557431 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:30.557390 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:30:30.557905 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:30.557621 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" containerID="cri-o://3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e" gracePeriod=30 Jun 08 18:30:32.604932 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.604904 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:32.671785 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.671752 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29k49\" (UniqueName: \"kubernetes.io/projected/f1e5624d-e0e7-4635-a95f-0d31d35e5704-kube-api-access-29k49\") pod \"f1e5624d-e0e7-4635-a95f-0d31d35e5704\" (UID: \"f1e5624d-e0e7-4635-a95f-0d31d35e5704\") " Jun 08 18:30:32.673956 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.673925 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1e5624d-e0e7-4635-a95f-0d31d35e5704-kube-api-access-29k49" (OuterVolumeSpecName: "kube-api-access-29k49") pod "f1e5624d-e0e7-4635-a95f-0d31d35e5704" (UID: "f1e5624d-e0e7-4635-a95f-0d31d35e5704"). InnerVolumeSpecName "kube-api-access-29k49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:30:32.772437 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.772356 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29k49\" (UniqueName: \"kubernetes.io/projected/f1e5624d-e0e7-4635-a95f-0d31d35e5704-kube-api-access-29k49\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:30:32.879900 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.879871 2560 generic.go:356] "Generic (PLEG): container finished" podID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerID="3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e" exitCode=143 Jun 08 18:30:32.880048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.879963 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:32.880048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.879973 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f1e5624d-e0e7-4635-a95f-0d31d35e5704","Type":"ContainerDied","Data":"3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e"} Jun 08 18:30:32.880048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.880011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f1e5624d-e0e7-4635-a95f-0d31d35e5704","Type":"ContainerDied","Data":"95cc909ad9d0576e27994473ff81acb6a48a4c8507dcfa80a047a4de1d3b884c"} Jun 08 18:30:32.880048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.880032 2560 scope.go:117] "RemoveContainer" containerID="3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e" Jun 08 18:30:32.889689 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.889674 2560 scope.go:117] "RemoveContainer" containerID="3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e" Jun 08 18:30:32.889963 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:30:32.889944 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e\": container with ID starting with 3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e not found: ID does not exist" containerID="3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e" Jun 08 18:30:32.890026 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.889969 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e"} err="failed to get container status \"3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e\": rpc error: code = NotFound desc = could not find container \"3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e\": container with ID starting with 3ecc16cbed0c7753d164cd04516aec1ee31563caaba729b7aa71bc83a901683e not found: ID does not exist" Jun 08 18:30:32.902582 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.902560 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:30:32.910511 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.910491 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:30:32.937108 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.937087 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:30:32.937444 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.937432 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" Jun 08 18:30:32.937444 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.937445 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" Jun 08 18:30:32.937519 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.937502 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" containerName="keycloak" Jun 08 18:30:32.941526 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.941512 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:32.944992 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.944972 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-qq2bh\"" Jun 08 18:30:32.945106 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.945046 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 08 18:30:32.945262 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.945247 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 08 18:30:32.945404 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.945376 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 08 18:30:32.945877 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.945837 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 08 18:30:32.955974 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.955953 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:30:32.973778 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.973755 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/4ddf9a59-ad45-4790-84bb-c67552a9b2ae-test-realms\") pod \"maas-keycloak-0\" (UID: \"4ddf9a59-ad45-4790-84bb-c67552a9b2ae\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:32.973884 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:32.973810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d58jf\" (UniqueName: \"kubernetes.io/projected/4ddf9a59-ad45-4790-84bb-c67552a9b2ae-kube-api-access-d58jf\") pod \"maas-keycloak-0\" (UID: \"4ddf9a59-ad45-4790-84bb-c67552a9b2ae\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:33.074517 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.074489 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/4ddf9a59-ad45-4790-84bb-c67552a9b2ae-test-realms\") pod \"maas-keycloak-0\" (UID: \"4ddf9a59-ad45-4790-84bb-c67552a9b2ae\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:33.074635 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.074574 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d58jf\" (UniqueName: \"kubernetes.io/projected/4ddf9a59-ad45-4790-84bb-c67552a9b2ae-kube-api-access-d58jf\") pod \"maas-keycloak-0\" (UID: \"4ddf9a59-ad45-4790-84bb-c67552a9b2ae\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:33.075247 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.075230 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/4ddf9a59-ad45-4790-84bb-c67552a9b2ae-test-realms\") pod \"maas-keycloak-0\" (UID: \"4ddf9a59-ad45-4790-84bb-c67552a9b2ae\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:33.084524 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.084502 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d58jf\" (UniqueName: \"kubernetes.io/projected/4ddf9a59-ad45-4790-84bb-c67552a9b2ae-kube-api-access-d58jf\") pod \"maas-keycloak-0\" (UID: \"4ddf9a59-ad45-4790-84bb-c67552a9b2ae\") " pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:33.250415 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.250383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:33.392624 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.392602 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 18:30:33.394962 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:30:33.394938 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ddf9a59_ad45_4790_84bb_c67552a9b2ae.slice/crio-8e1c36e3655c5662a245609a997f53616c0f8d422cf1b470606440815f1b9dde WatchSource:0}: Error finding container 8e1c36e3655c5662a245609a997f53616c0f8d422cf1b470606440815f1b9dde: Status 404 returned error can't find the container with id 8e1c36e3655c5662a245609a997f53616c0f8d422cf1b470606440815f1b9dde Jun 08 18:30:33.841464 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.841440 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-p5fnj" Jun 08 18:30:33.841775 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.841521 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-9m7hr" Jun 08 18:30:33.890108 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.890066 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"4ddf9a59-ad45-4790-84bb-c67552a9b2ae","Type":"ContainerStarted","Data":"8b5c36d2779385a582310c35051a8c2eec2f667d15a3f9fe16c3904a8009bb94"} Jun 08 18:30:33.890257 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.890123 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"4ddf9a59-ad45-4790-84bb-c67552a9b2ae","Type":"ContainerStarted","Data":"8e1c36e3655c5662a245609a997f53616c0f8d422cf1b470606440815f1b9dde"} Jun 08 18:30:33.915007 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:33.914957 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.598190757 podStartE2EDuration="1.914938678s" totalImagesPullingTime="316.747921ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:30:33.914227903 +0000 UTC m=+908.075064562" watchObservedRunningTime="2026-06-08 18:30:33.914938678 +0000 UTC m=+908.075775336" Jun 08 18:30:34.250749 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:34.250709 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:34.252110 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:34.252079 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:34.338567 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:34.338530 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1e5624d-e0e7-4635-a95f-0d31d35e5704" path="/var/lib/kubelet/pods/f1e5624d-e0e7-4635-a95f-0d31d35e5704/volumes" Jun 08 18:30:35.251799 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:35.251745 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:36.251516 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:36.251463 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:37.251786 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:37.251730 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:38.251832 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:38.251734 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:39.250911 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:39.250861 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:40.251455 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:40.251394 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:41.251335 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:41.251283 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:42.251468 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:42.251418 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:43.250725 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:43.250685 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:43.251005 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:43.250977 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:44.251609 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:44.251553 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:45.251792 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:45.251744 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:46.251746 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:46.251708 2560 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.40:9000/health/started\": dial tcp 10.133.0.40:9000: connect: connection refused" Jun 08 18:30:47.376395 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:47.376357 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 08 18:30:47.393504 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:47.393459 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="4ddf9a59-ad45-4790-84bb-c67552a9b2ae" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:30:57.382445 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:30:57.382405 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 08 18:31:18.598562 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.598530 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6"] Jun 08 18:31:18.601985 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.601969 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.605707 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.605686 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 08 18:31:18.605837 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.605687 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 08 18:31:18.605837 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.605729 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mpm57\"" Jun 08 18:31:18.606019 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.606006 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 08 18:31:18.615580 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.615556 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6"] Jun 08 18:31:18.708915 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.708878 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.708915 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.708921 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g65fp\" (UniqueName: \"kubernetes.io/projected/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-kube-api-access-g65fp\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.709162 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.708947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.709162 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.708964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.709162 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.709052 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.709162 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.709101 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810244 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810211 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g65fp\" (UniqueName: \"kubernetes.io/projected/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-kube-api-access-g65fp\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810433 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810433 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810433 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810433 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810638 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810724 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810704 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810804 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810780 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.810895 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.810810 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.812769 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.812744 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.812972 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.812954 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.818551 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.818532 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g65fp\" (UniqueName: \"kubernetes.io/projected/56d65467-3d6f-4ce4-8ce7-d0f967a0708a-kube-api-access-g65fp\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-7pqr6\" (UID: \"56d65467-3d6f-4ce4-8ce7-d0f967a0708a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:18.912470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:18.912403 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" Jun 08 18:31:19.080658 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:19.080634 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6"] Jun 08 18:31:19.082208 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:31:19.082178 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d65467_3d6f_4ce4_8ce7_d0f967a0708a.slice/crio-d3f9237a8c140b228ef8967712ae50cc824b870b95ab6c979b8e98de5eae2af9 WatchSource:0}: Error finding container d3f9237a8c140b228ef8967712ae50cc824b870b95ab6c979b8e98de5eae2af9: Status 404 returned error can't find the container with id d3f9237a8c140b228ef8967712ae50cc824b870b95ab6c979b8e98de5eae2af9 Jun 08 18:31:19.168077 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:19.167984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" event={"ID":"56d65467-3d6f-4ce4-8ce7-d0f967a0708a","Type":"ContainerStarted","Data":"d3f9237a8c140b228ef8967712ae50cc824b870b95ab6c979b8e98de5eae2af9"} Jun 08 18:31:23.191571 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:23.191530 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-7pqr6" event={"ID":"56d65467-3d6f-4ce4-8ce7-d0f967a0708a","Type":"ContainerStarted","Data":"73b4d5532b1938e937336cb9da83c7183f6531dd4b39031cba6f0c2315aef9e9"} Jun 08 18:31:57.073319 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.073237 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54"] Jun 08 18:31:57.083803 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.083776 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.088019 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.087991 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 08 18:31:57.090630 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.090607 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54"] Jun 08 18:31:57.149136 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.149096 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5rgk\" (UniqueName: \"kubernetes.io/projected/290a5523-db87-457d-8b10-70d6531b77fe-kube-api-access-r5rgk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.149136 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.149152 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.149398 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.149276 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.149398 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.149315 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/290a5523-db87-457d-8b10-70d6531b77fe-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.149398 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.149377 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.149556 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.149411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.250437 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.250403 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.250437 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.250446 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/290a5523-db87-457d-8b10-70d6531b77fe-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.250669 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.250597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.250669 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.250637 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.250795 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.250685 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5rgk\" (UniqueName: \"kubernetes.io/projected/290a5523-db87-457d-8b10-70d6531b77fe-kube-api-access-r5rgk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.250795 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.250712 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.250952 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.250870 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.251014 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.250998 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.251089 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.251069 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.252946 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.252927 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/290a5523-db87-457d-8b10-70d6531b77fe-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.253140 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.253120 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/290a5523-db87-457d-8b10-70d6531b77fe-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.258271 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.258246 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5rgk\" (UniqueName: \"kubernetes.io/projected/290a5523-db87-457d-8b10-70d6531b77fe-kube-api-access-r5rgk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54\" (UID: \"290a5523-db87-457d-8b10-70d6531b77fe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.395023 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.394936 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:31:57.533990 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:57.533968 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54"] Jun 08 18:31:57.536480 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:31:57.536453 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290a5523_db87_457d_8b10_70d6531b77fe.slice/crio-a2ea0ac31641855bfac4c03966a772af89905a1607cc55377ebd986e2163941a WatchSource:0}: Error finding container a2ea0ac31641855bfac4c03966a772af89905a1607cc55377ebd986e2163941a: Status 404 returned error can't find the container with id a2ea0ac31641855bfac4c03966a772af89905a1607cc55377ebd986e2163941a Jun 08 18:31:58.361471 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:58.361437 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" event={"ID":"290a5523-db87-457d-8b10-70d6531b77fe","Type":"ContainerStarted","Data":"eb8f99270383fdf7513ede3426753769b0de72d05c0ee59868866c4f863fe0bf"} Jun 08 18:31:58.361829 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:58.361493 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" event={"ID":"290a5523-db87-457d-8b10-70d6531b77fe","Type":"ContainerStarted","Data":"a2ea0ac31641855bfac4c03966a772af89905a1607cc55377ebd986e2163941a"} Jun 08 18:31:59.367086 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:59.367057 2560 generic.go:356] "Generic (PLEG): container finished" podID="290a5523-db87-457d-8b10-70d6531b77fe" containerID="eb8f99270383fdf7513ede3426753769b0de72d05c0ee59868866c4f863fe0bf" exitCode=0 Jun 08 18:31:59.367465 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:31:59.367144 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" event={"ID":"290a5523-db87-457d-8b10-70d6531b77fe","Type":"ContainerDied","Data":"eb8f99270383fdf7513ede3426753769b0de72d05c0ee59868866c4f863fe0bf"} Jun 08 18:32:01.379684 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:01.379644 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" event={"ID":"290a5523-db87-457d-8b10-70d6531b77fe","Type":"ContainerStarted","Data":"54d170899d760c21b5035112877402cba9050b80a9b50f3da72df801ae5ffd7c"} Jun 08 18:32:01.399199 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:01.399145 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" podStartSLOduration=1.9659213260000001 podStartE2EDuration="4.399128229s" totalImagesPullingTime="1.1531381s" totalInitContainerRuntime="1.280068803s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:31:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:32:01.397727887 +0000 UTC m=+995.558564545" watchObservedRunningTime="2026-06-08 18:32:01.399128229 +0000 UTC m=+995.559964886" Jun 08 18:32:07.395957 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:07.395918 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:32:07.396343 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:07.396021 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:32:07.408752 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:07.408726 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:32:08.423977 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:08.423949 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54" Jun 08 18:32:17.770397 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.770361 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc"] Jun 08 18:32:17.774685 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.774659 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.777526 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.777505 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 08 18:32:17.792324 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.792297 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc"] Jun 08 18:32:17.839565 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.839533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.839565 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.839562 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.839708 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.839583 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/879fecf5-6677-4475-a4eb-96a826784aa2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.839708 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.839606 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fh5j\" (UniqueName: \"kubernetes.io/projected/879fecf5-6677-4475-a4eb-96a826784aa2-kube-api-access-6fh5j\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.839780 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.839729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.839780 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.839769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940292 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940292 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940536 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940536 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940536 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940386 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/879fecf5-6677-4475-a4eb-96a826784aa2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940536 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fh5j\" (UniqueName: \"kubernetes.io/projected/879fecf5-6677-4475-a4eb-96a826784aa2-kube-api-access-6fh5j\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940759 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940736 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940826 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940744 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.940916 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.940839 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.943334 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.943301 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/879fecf5-6677-4475-a4eb-96a826784aa2-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.943450 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.943388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/879fecf5-6677-4475-a4eb-96a826784aa2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:17.950717 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:17.950693 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fh5j\" (UniqueName: \"kubernetes.io/projected/879fecf5-6677-4475-a4eb-96a826784aa2-kube-api-access-6fh5j\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-tvfmc\" (UID: \"879fecf5-6677-4475-a4eb-96a826784aa2\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:18.084312 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:18.084280 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:32:18.229466 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:18.229440 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc"] Jun 08 18:32:18.231864 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:32:18.231809 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879fecf5_6677_4475_a4eb_96a826784aa2.slice/crio-eea78dd8bb57368808fa340d9ef4e52b3e9da0820abb9cbf70d04eaec2a4d93f WatchSource:0}: Error finding container eea78dd8bb57368808fa340d9ef4e52b3e9da0820abb9cbf70d04eaec2a4d93f: Status 404 returned error can't find the container with id eea78dd8bb57368808fa340d9ef4e52b3e9da0820abb9cbf70d04eaec2a4d93f Jun 08 18:32:18.463654 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:18.463568 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" event={"ID":"879fecf5-6677-4475-a4eb-96a826784aa2","Type":"ContainerStarted","Data":"86ffdac3151d831fc1822493c39a8b4c3cdae2ef4a34dc4dbb5134412269d8bc"} Jun 08 18:32:18.463654 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:32:18.463617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" event={"ID":"879fecf5-6677-4475-a4eb-96a826784aa2","Type":"ContainerStarted","Data":"eea78dd8bb57368808fa340d9ef4e52b3e9da0820abb9cbf70d04eaec2a4d93f"} Jun 08 18:33:12.725122 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:12.725089 2560 generic.go:356] "Generic (PLEG): container finished" podID="879fecf5-6677-4475-a4eb-96a826784aa2" containerID="86ffdac3151d831fc1822493c39a8b4c3cdae2ef4a34dc4dbb5134412269d8bc" exitCode=0 Jun 08 18:33:12.725582 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:12.725165 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" event={"ID":"879fecf5-6677-4475-a4eb-96a826784aa2","Type":"ContainerDied","Data":"86ffdac3151d831fc1822493c39a8b4c3cdae2ef4a34dc4dbb5134412269d8bc"} Jun 08 18:33:12.725797 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:12.725781 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 18:33:13.731897 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:13.731809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" event={"ID":"879fecf5-6677-4475-a4eb-96a826784aa2","Type":"ContainerStarted","Data":"93c9228dc6cd94087def0b18d7a32c5a226381648c0a1b4b848a2fde025c1607"} Jun 08 18:33:13.755176 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:13.755132 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" podStartSLOduration=2.426555537 podStartE2EDuration="56.75511572s" totalImagesPullingTime="327.011337ms" totalInitContainerRuntime="54.001548846s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:33:13.753807749 +0000 UTC m=+1067.914644433" watchObservedRunningTime="2026-06-08 18:33:13.75511572 +0000 UTC m=+1067.915952419" Jun 08 18:33:18.084746 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:18.084713 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:33:18.085251 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:18.084784 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:33:18.096835 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:18.096811 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:33:18.768730 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:33:18.768705 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-tvfmc" Jun 08 18:35:12.503179 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.503088 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cb9f9847-hd869"] Jun 08 18:35:12.503770 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.503347 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6cb9f9847-hd869" podUID="2d42a19e-882c-4c11-86c9-c3e72a380c83" containerName="manager" containerID="cri-o://e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732" gracePeriod=30 Jun 08 18:35:12.760997 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.760938 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:35:12.853030 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.853000 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vttf4\" (UniqueName: \"kubernetes.io/projected/2d42a19e-882c-4c11-86c9-c3e72a380c83-kube-api-access-vttf4\") pod \"2d42a19e-882c-4c11-86c9-c3e72a380c83\" (UID: \"2d42a19e-882c-4c11-86c9-c3e72a380c83\") " Jun 08 18:35:12.853197 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.853066 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d42a19e-882c-4c11-86c9-c3e72a380c83-webhook-cert\") pod \"2d42a19e-882c-4c11-86c9-c3e72a380c83\" (UID: \"2d42a19e-882c-4c11-86c9-c3e72a380c83\") " Jun 08 18:35:12.855075 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.855048 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d42a19e-882c-4c11-86c9-c3e72a380c83-kube-api-access-vttf4" (OuterVolumeSpecName: "kube-api-access-vttf4") pod "2d42a19e-882c-4c11-86c9-c3e72a380c83" (UID: "2d42a19e-882c-4c11-86c9-c3e72a380c83"). InnerVolumeSpecName "kube-api-access-vttf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:35:12.855193 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.855149 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d42a19e-882c-4c11-86c9-c3e72a380c83-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2d42a19e-882c-4c11-86c9-c3e72a380c83" (UID: "2d42a19e-882c-4c11-86c9-c3e72a380c83"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:35:12.954251 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.954224 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vttf4\" (UniqueName: \"kubernetes.io/projected/2d42a19e-882c-4c11-86c9-c3e72a380c83-kube-api-access-vttf4\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:35:12.954251 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:12.954249 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d42a19e-882c-4c11-86c9-c3e72a380c83-webhook-cert\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:35:13.330635 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.330599 2560 generic.go:356] "Generic (PLEG): container finished" podID="2d42a19e-882c-4c11-86c9-c3e72a380c83" containerID="e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732" exitCode=0 Jun 08 18:35:13.330635 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.330638 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb9f9847-hd869" Jun 08 18:35:13.330906 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.330683 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb9f9847-hd869" event={"ID":"2d42a19e-882c-4c11-86c9-c3e72a380c83","Type":"ContainerDied","Data":"e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732"} Jun 08 18:35:13.330906 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.330737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb9f9847-hd869" event={"ID":"2d42a19e-882c-4c11-86c9-c3e72a380c83","Type":"ContainerDied","Data":"8d1cbd1bf447a2b479ff706fe5952e1256d9cb50a92355bc4bd3f38012959055"} Jun 08 18:35:13.330906 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.330759 2560 scope.go:117] "RemoveContainer" containerID="e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732" Jun 08 18:35:13.340091 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.340076 2560 scope.go:117] "RemoveContainer" containerID="e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732" Jun 08 18:35:13.340343 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:35:13.340325 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732\": container with ID starting with e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732 not found: ID does not exist" containerID="e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732" Jun 08 18:35:13.340403 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.340351 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732"} err="failed to get container status \"e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732\": rpc error: code = NotFound desc = could not find container \"e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732\": container with ID starting with e756dff7f5bf39c8317092eeda78c1cc98c3d7f1c136103a300ea0f27a10f732 not found: ID does not exist" Jun 08 18:35:13.351710 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.351688 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cb9f9847-hd869"] Jun 08 18:35:13.355347 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:13.355328 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cb9f9847-hd869"] Jun 08 18:35:14.120666 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.120636 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cb9f9847-bz9jn"] Jun 08 18:35:14.121068 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.121048 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d42a19e-882c-4c11-86c9-c3e72a380c83" containerName="manager" Jun 08 18:35:14.121068 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.121060 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d42a19e-882c-4c11-86c9-c3e72a380c83" containerName="manager" Jun 08 18:35:14.121140 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.121134 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d42a19e-882c-4c11-86c9-c3e72a380c83" containerName="manager" Jun 08 18:35:14.125470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.125454 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:14.129391 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.129372 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 18:35:14.129519 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.129412 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 18:35:14.129519 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.129453 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-25l86\"" Jun 08 18:35:14.135062 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.135039 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb9f9847-bz9jn"] Jun 08 18:35:14.165098 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.165071 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpt46\" (UniqueName: \"kubernetes.io/projected/e7fed784-1336-4a75-be7f-0b5f17043b2e-kube-api-access-mpt46\") pod \"maas-controller-6cb9f9847-bz9jn\" (UID: \"e7fed784-1336-4a75-be7f-0b5f17043b2e\") " pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:14.165202 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.165121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7fed784-1336-4a75-be7f-0b5f17043b2e-webhook-cert\") pod \"maas-controller-6cb9f9847-bz9jn\" (UID: \"e7fed784-1336-4a75-be7f-0b5f17043b2e\") " pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:14.266456 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.266430 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpt46\" (UniqueName: \"kubernetes.io/projected/e7fed784-1336-4a75-be7f-0b5f17043b2e-kube-api-access-mpt46\") pod \"maas-controller-6cb9f9847-bz9jn\" (UID: \"e7fed784-1336-4a75-be7f-0b5f17043b2e\") " pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:14.266543 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.266481 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7fed784-1336-4a75-be7f-0b5f17043b2e-webhook-cert\") pod \"maas-controller-6cb9f9847-bz9jn\" (UID: \"e7fed784-1336-4a75-be7f-0b5f17043b2e\") " pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:14.268997 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.268979 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7fed784-1336-4a75-be7f-0b5f17043b2e-webhook-cert\") pod \"maas-controller-6cb9f9847-bz9jn\" (UID: \"e7fed784-1336-4a75-be7f-0b5f17043b2e\") " pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:14.274281 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.274263 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpt46\" (UniqueName: \"kubernetes.io/projected/e7fed784-1336-4a75-be7f-0b5f17043b2e-kube-api-access-mpt46\") pod \"maas-controller-6cb9f9847-bz9jn\" (UID: \"e7fed784-1336-4a75-be7f-0b5f17043b2e\") " pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:14.337594 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.337563 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d42a19e-882c-4c11-86c9-c3e72a380c83" path="/var/lib/kubelet/pods/2d42a19e-882c-4c11-86c9-c3e72a380c83/volumes" Jun 08 18:35:14.436825 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.436767 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:14.570101 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:14.570078 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb9f9847-bz9jn"] Jun 08 18:35:14.572289 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:35:14.572263 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fed784_1336_4a75_be7f_0b5f17043b2e.slice/crio-cd3ec8a6f9fb2bdbb584fb7653146c7773ea95b2c1b7119f6aeed059268d2f5d WatchSource:0}: Error finding container cd3ec8a6f9fb2bdbb584fb7653146c7773ea95b2c1b7119f6aeed059268d2f5d: Status 404 returned error can't find the container with id cd3ec8a6f9fb2bdbb584fb7653146c7773ea95b2c1b7119f6aeed059268d2f5d Jun 08 18:35:15.343016 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:15.342988 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb9f9847-bz9jn" event={"ID":"e7fed784-1336-4a75-be7f-0b5f17043b2e","Type":"ContainerStarted","Data":"bf5314d5c17a683f7fedc00eb6ba41caf80d1f8bf87958058c477f989da160b8"} Jun 08 18:35:15.343382 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:15.343033 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb9f9847-bz9jn" event={"ID":"e7fed784-1336-4a75-be7f-0b5f17043b2e","Type":"ContainerStarted","Data":"cd3ec8a6f9fb2bdbb584fb7653146c7773ea95b2c1b7119f6aeed059268d2f5d"} Jun 08 18:35:15.343382 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:15.343064 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:15.361667 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:15.361561 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cb9f9847-bz9jn" podStartSLOduration=0.949647228 podStartE2EDuration="1.361542873s" totalImagesPullingTime="411.895645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:35:15.360871241 +0000 UTC m=+1189.521707890" watchObservedRunningTime="2026-06-08 18:35:15.361542873 +0000 UTC m=+1189.522379531" Jun 08 18:35:26.353181 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:26.353151 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cb9f9847-bz9jn" Jun 08 18:35:26.388316 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:26.388294 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:35:26.390332 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:35:26.390306 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:40:26.420931 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:40:26.420902 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:40:26.423227 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:40:26.423209 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:45:26.457527 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:26.457502 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:45:26.464428 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:26.464407 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:45:46.673467 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:46.673433 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr"] Jun 08 18:45:46.673928 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:46.673627 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" podUID="3109d910-1b1c-4a16-b55c-ab694f133abe" containerName="manager" containerID="cri-o://d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62" gracePeriod=10 Jun 08 18:45:46.933953 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:46.933891 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:45:47.085554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.085505 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3109d910-1b1c-4a16-b55c-ab694f133abe-extensions-socket-volume\") pod \"3109d910-1b1c-4a16-b55c-ab694f133abe\" (UID: \"3109d910-1b1c-4a16-b55c-ab694f133abe\") " Jun 08 18:45:47.085751 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.085581 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qb9dm\" (UniqueName: \"kubernetes.io/projected/3109d910-1b1c-4a16-b55c-ab694f133abe-kube-api-access-qb9dm\") pod \"3109d910-1b1c-4a16-b55c-ab694f133abe\" (UID: \"3109d910-1b1c-4a16-b55c-ab694f133abe\") " Jun 08 18:45:47.085911 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.085883 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3109d910-1b1c-4a16-b55c-ab694f133abe-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3109d910-1b1c-4a16-b55c-ab694f133abe" (UID: "3109d910-1b1c-4a16-b55c-ab694f133abe"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:45:47.085977 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.085964 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3109d910-1b1c-4a16-b55c-ab694f133abe-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:45:47.087739 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.087715 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3109d910-1b1c-4a16-b55c-ab694f133abe-kube-api-access-qb9dm" (OuterVolumeSpecName: "kube-api-access-qb9dm") pod "3109d910-1b1c-4a16-b55c-ab694f133abe" (UID: "3109d910-1b1c-4a16-b55c-ab694f133abe"). InnerVolumeSpecName "kube-api-access-qb9dm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:45:47.186699 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.186617 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb9dm\" (UniqueName: \"kubernetes.io/projected/3109d910-1b1c-4a16-b55c-ab694f133abe-kube-api-access-qb9dm\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jun 08 18:45:47.447741 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.447643 2560 generic.go:356] "Generic (PLEG): container finished" podID="3109d910-1b1c-4a16-b55c-ab694f133abe" containerID="d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62" exitCode=0 Jun 08 18:45:47.447741 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.447700 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" Jun 08 18:45:47.447741 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.447718 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" event={"ID":"3109d910-1b1c-4a16-b55c-ab694f133abe","Type":"ContainerDied","Data":"d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62"} Jun 08 18:45:47.448051 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.447814 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr" event={"ID":"3109d910-1b1c-4a16-b55c-ab694f133abe","Type":"ContainerDied","Data":"fe11e3034c022544ee10509c0c5b8a049f2c10668d714b2af52ced0395657649"} Jun 08 18:45:47.448051 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.447909 2560 scope.go:117] "RemoveContainer" containerID="d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62" Jun 08 18:45:47.457405 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.457387 2560 scope.go:117] "RemoveContainer" containerID="d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62" Jun 08 18:45:47.457647 ip-10-0-131-227 kubenswrapper[2560]: E0608 18:45:47.457628 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62\": container with ID starting with d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62 not found: ID does not exist" containerID="d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62" Jun 08 18:45:47.457698 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.457656 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62"} err="failed to get container status \"d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62\": rpc error: code = NotFound desc = could not find container \"d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62\": container with ID starting with d7ebe37cde5e61bc1c422ef5a0454373997ed5935cfa9f7376c060f721ec5a62 not found: ID does not exist" Jun 08 18:45:47.519538 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.519509 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr"] Jun 08 18:45:47.532991 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:47.532966 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-464nr"] Jun 08 18:45:48.338461 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:45:48.338427 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3109d910-1b1c-4a16-b55c-ab694f133abe" path="/var/lib/kubelet/pods/3109d910-1b1c-4a16-b55c-ab694f133abe/volumes" Jun 08 18:46:52.814274 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.814181 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2"] Jun 08 18:46:52.814690 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.814635 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3109d910-1b1c-4a16-b55c-ab694f133abe" containerName="manager" Jun 08 18:46:52.814690 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.814649 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3109d910-1b1c-4a16-b55c-ab694f133abe" containerName="manager" Jun 08 18:46:52.814784 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.814720 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3109d910-1b1c-4a16-b55c-ab694f133abe" containerName="manager" Jun 08 18:46:52.818202 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.818185 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:52.821333 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.821305 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sknbj\"" Jun 08 18:46:52.830322 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.830289 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2"] Jun 08 18:46:52.875921 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.875880 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pwzr\" (UniqueName: \"kubernetes.io/projected/8b492201-093e-44dd-9b36-334959b3f329-kube-api-access-4pwzr\") pod \"kuadrant-operator-controller-manager-86878c769f-qmlm2\" (UID: \"8b492201-093e-44dd-9b36-334959b3f329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:52.876095 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.875999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b492201-093e-44dd-9b36-334959b3f329-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qmlm2\" (UID: \"8b492201-093e-44dd-9b36-334959b3f329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:52.977206 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.977167 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pwzr\" (UniqueName: \"kubernetes.io/projected/8b492201-093e-44dd-9b36-334959b3f329-kube-api-access-4pwzr\") pod \"kuadrant-operator-controller-manager-86878c769f-qmlm2\" (UID: \"8b492201-093e-44dd-9b36-334959b3f329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:52.977372 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.977259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b492201-093e-44dd-9b36-334959b3f329-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qmlm2\" (UID: \"8b492201-093e-44dd-9b36-334959b3f329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:52.977602 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.977586 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b492201-093e-44dd-9b36-334959b3f329-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qmlm2\" (UID: \"8b492201-093e-44dd-9b36-334959b3f329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:52.990714 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:52.990681 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pwzr\" (UniqueName: \"kubernetes.io/projected/8b492201-093e-44dd-9b36-334959b3f329-kube-api-access-4pwzr\") pod \"kuadrant-operator-controller-manager-86878c769f-qmlm2\" (UID: \"8b492201-093e-44dd-9b36-334959b3f329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:53.128549 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:53.128466 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:53.274566 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:53.274538 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2"] Jun 08 18:46:53.276952 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:46:53.276920 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b492201_093e_44dd_9b36_334959b3f329.slice/crio-dd44a24198a5916034e74b801d4a99b1aafee4ec00997cef81aa866bab0838b1 WatchSource:0}: Error finding container dd44a24198a5916034e74b801d4a99b1aafee4ec00997cef81aa866bab0838b1: Status 404 returned error can't find the container with id dd44a24198a5916034e74b801d4a99b1aafee4ec00997cef81aa866bab0838b1 Jun 08 18:46:53.772275 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:53.772240 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" event={"ID":"8b492201-093e-44dd-9b36-334959b3f329","Type":"ContainerStarted","Data":"9b09504f2fb97bb3f803dd91a156120cc80aff7fbcda4eb6024362693f782837"} Jun 08 18:46:53.772483 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:53.772297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" event={"ID":"8b492201-093e-44dd-9b36-334959b3f329","Type":"ContainerStarted","Data":"dd44a24198a5916034e74b801d4a99b1aafee4ec00997cef81aa866bab0838b1"} Jun 08 18:46:53.772483 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:53.772345 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:46:53.796255 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:46:53.796205 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" podStartSLOduration=1.796190158 podStartE2EDuration="1.796190158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:46:53.794388653 +0000 UTC m=+1887.955225326" watchObservedRunningTime="2026-06-08 18:46:53.796190158 +0000 UTC m=+1887.957026815" Jun 08 18:47:04.779820 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:47:04.779791 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qmlm2" Jun 08 18:50:26.489452 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:50:26.489431 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:50:26.497683 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:50:26.497664 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:55:26.520130 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:55:26.520106 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:55:26.529971 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:55:26.529949 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:56:01.168480 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:01.168410 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-84b585c54b-5vkjs_b8c3624d-0ecd-407e-ba87-c45384ad36be/nginx/0.log" Jun 08 18:56:01.626148 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:01.626098 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-77d5447464-822qr_f9bca354-5115-46fb-b6de-3981ba703534/maas-api/0.log" Jun 08 18:56:01.852018 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:01.851993 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cb9f9847-bz9jn_e7fed784-1336-4a75-be7f-0b5f17043b2e/manager/0.log" Jun 08 18:56:02.482634 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:02.482606 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4695b4ff-lxvrm_6f9e5797-8a69-4c8b-af51-6271c1ae2ea3/manager/0.log" Jun 08 18:56:02.496554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:02.496530 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4695b4ff-lxvrm_6f9e5797-8a69-4c8b-af51-6271c1ae2ea3/copy-manifests/0.log" Jun 08 18:56:03.994521 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:03.994490 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wqpb5_d45b58d3-44a6-47bf-b27b-028d3760041c/manager/0.log" Jun 08 18:56:04.100488 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:04.100462 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-fcm9c_54b0b175-594b-4a50-92f4-c9f46a75913d/manager/0.log" Jun 08 18:56:04.324636 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:04.324611 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2zk8s_cb20b223-e9e7-4ec3-92fc-7f7e8fb15873/registry-server/0.log" Jun 08 18:56:04.454310 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:04.454282 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qmlm2_8b492201-093e-44dd-9b36-334959b3f329/manager/0.log" Jun 08 18:56:04.701389 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:04.701314 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2nwld_c8eeaa7a-6c43-419b-bccb-f846603904fa/manager/0.log" Jun 08 18:56:05.034324 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:05.034257 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795lsj7w_f04b0cf9-3011-4541-92bf-49510e2699d0/istio-proxy/0.log" Jun 08 18:56:05.588164 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:05.588140 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p5fnj_a23ba43d-8f9b-47f1-a68c-b175c25bdc95/payload-pre-processing/0.log" Jun 08 18:56:05.706081 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:05.706054 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9m7hr_74581d47-cce4-49b4-a980-0cc0e48c05ce/payload-processing/0.log" Jun 08 18:56:06.492240 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:06.492214 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54_290a5523-db87-457d-8b10-70d6531b77fe/storage-initializer/0.log" Jun 08 18:56:06.500715 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:06.500688 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c89p54_290a5523-db87-457d-8b10-70d6531b77fe/main/0.log" Jun 08 18:56:06.751750 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:06.751675 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-tvfmc_879fecf5-6677-4475-a4eb-96a826784aa2/storage-initializer/0.log" Jun 08 18:56:06.759354 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:06.759332 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-tvfmc_879fecf5-6677-4475-a4eb-96a826784aa2/main/0.log" Jun 08 18:56:13.567009 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:13.566984 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lbhcf_796c82e4-e1bb-436b-9145-8377aed43b06/global-pull-secret-syncer/0.log" Jun 08 18:56:13.672802 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:13.672775 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vl77q_ce0ccf9f-a60b-4b52-8e2a-e4a420c337fa/konnectivity-agent/0.log" Jun 08 18:56:13.693836 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:13.693801 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-227.ec2.internal_b0bdb72680f38d3f1fa35164d90fd6e0/haproxy/0.log" Jun 08 18:56:18.729663 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:18.729627 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wqpb5_d45b58d3-44a6-47bf-b27b-028d3760041c/manager/0.log" Jun 08 18:56:18.760076 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:18.760047 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-fcm9c_54b0b175-594b-4a50-92f4-c9f46a75913d/manager/0.log" Jun 08 18:56:18.820776 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:18.820743 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2zk8s_cb20b223-e9e7-4ec3-92fc-7f7e8fb15873/registry-server/0.log" Jun 08 18:56:18.876086 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:18.876059 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qmlm2_8b492201-093e-44dd-9b36-334959b3f329/manager/0.log" Jun 08 18:56:18.961789 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:18.961760 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2nwld_c8eeaa7a-6c43-419b-bccb-f846603904fa/manager/0.log" Jun 08 18:56:21.106359 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.106327 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5bz8_cf63ce1b-9335-49ab-a593-a094552a3d35/node-exporter/0.log" Jun 08 18:56:21.134909 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.134825 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5bz8_cf63ce1b-9335-49ab-a593-a094552a3d35/kube-rbac-proxy/0.log" Jun 08 18:56:21.156329 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.156299 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5bz8_cf63ce1b-9335-49ab-a593-a094552a3d35/init-textfile/0.log" Jun 08 18:56:21.262772 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.262749 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_740056a9-3a16-425d-b597-8bb7a324d936/prometheus/0.log" Jun 08 18:56:21.283678 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.283654 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_740056a9-3a16-425d-b597-8bb7a324d936/config-reloader/0.log" Jun 08 18:56:21.305530 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.305505 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_740056a9-3a16-425d-b597-8bb7a324d936/thanos-sidecar/0.log" Jun 08 18:56:21.326903 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.326881 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_740056a9-3a16-425d-b597-8bb7a324d936/kube-rbac-proxy-web/0.log" Jun 08 18:56:21.346822 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.346796 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_740056a9-3a16-425d-b597-8bb7a324d936/kube-rbac-proxy/0.log" Jun 08 18:56:21.368979 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.368952 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_740056a9-3a16-425d-b597-8bb7a324d936/kube-rbac-proxy-thanos/0.log" Jun 08 18:56:21.390487 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.390419 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_740056a9-3a16-425d-b597-8bb7a324d936/init-config-reloader/0.log" Jun 08 18:56:21.604949 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.604915 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dbb7db6ff-rrxhc_3be6b919-a677-4ab5-b8c7-11d35f463b79/thanos-query/0.log" Jun 08 18:56:21.627273 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.627243 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dbb7db6ff-rrxhc_3be6b919-a677-4ab5-b8c7-11d35f463b79/kube-rbac-proxy-web/0.log" Jun 08 18:56:21.649485 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.649392 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dbb7db6ff-rrxhc_3be6b919-a677-4ab5-b8c7-11d35f463b79/kube-rbac-proxy/0.log" Jun 08 18:56:21.673236 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.673204 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dbb7db6ff-rrxhc_3be6b919-a677-4ab5-b8c7-11d35f463b79/prom-label-proxy/0.log" Jun 08 18:56:21.694216 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.694188 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dbb7db6ff-rrxhc_3be6b919-a677-4ab5-b8c7-11d35f463b79/kube-rbac-proxy-rules/0.log" Jun 08 18:56:21.717470 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:21.717444 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6dbb7db6ff-rrxhc_3be6b919-a677-4ab5-b8c7-11d35f463b79/kube-rbac-proxy-metrics/0.log" Jun 08 18:56:23.244401 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:23.244369 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-rjsgl_ce0b5be3-c42e-4ae3-93d2-18244fb5bf36/networking-console-plugin/0.log" Jun 08 18:56:25.221554 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.221514 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj"] Jun 08 18:56:25.225675 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.225652 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.228368 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.228347 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2nmtb\"/\"default-dockercfg-xqxmt\"" Jun 08 18:56:25.229382 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.229362 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2nmtb\"/\"kube-root-ca.crt\"" Jun 08 18:56:25.229492 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.229410 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2nmtb\"/\"openshift-service-ca.crt\"" Jun 08 18:56:25.233229 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.233207 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj"] Jun 08 18:56:25.311857 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.311814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-sys\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.312012 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.311894 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8n7t\" (UniqueName: \"kubernetes.io/projected/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-kube-api-access-r8n7t\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.312012 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.311984 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-podres\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.312111 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.312022 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-proc\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.312111 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.312087 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-lib-modules\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413201 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-sys\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413201 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8n7t\" (UniqueName: \"kubernetes.io/projected/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-kube-api-access-r8n7t\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413426 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-podres\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413426 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-proc\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413426 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-lib-modules\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413426 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413290 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-sys\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413426 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413361 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-proc\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413426 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413398 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-podres\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.413426 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.413409 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-lib-modules\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.423044 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.423018 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8n7t\" (UniqueName: \"kubernetes.io/projected/836ab0f5-0220-42dd-9f42-c46c30a7f9ca-kube-api-access-r8n7t\") pod \"perf-node-gather-daemonset-sn2vj\" (UID: \"836ab0f5-0220-42dd-9f42-c46c30a7f9ca\") " pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.536755 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.536661 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:25.678565 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:25.678534 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj"] Jun 08 18:56:25.680200 ip-10-0-131-227 kubenswrapper[2560]: W0608 18:56:25.680173 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod836ab0f5_0220_42dd_9f42_c46c30a7f9ca.slice/crio-fe001edfbd779229f27adca7f7d762f5fa675aef41ddfaf16b72faf9093fc21e WatchSource:0}: Error finding container fe001edfbd779229f27adca7f7d762f5fa675aef41ddfaf16b72faf9093fc21e: Status 404 returned error can't find the container with id fe001edfbd779229f27adca7f7d762f5fa675aef41ddfaf16b72faf9093fc21e Jun 08 18:56:26.428524 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:26.428494 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l7b8f_781da7a9-9004-4d56-897b-de4d48846c9f/dns/0.log" Jun 08 18:56:26.451856 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:26.451811 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l7b8f_781da7a9-9004-4d56-897b-de4d48846c9f/kube-rbac-proxy/0.log" Jun 08 18:56:26.574677 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:26.574644 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mqbnh_02733b0c-c0c5-422b-8d01-de38f405ef33/dns-node-resolver/0.log" Jun 08 18:56:26.636360 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:26.636320 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" event={"ID":"836ab0f5-0220-42dd-9f42-c46c30a7f9ca","Type":"ContainerStarted","Data":"60ac0d9e79586c96347b0a11aad46029ce509e57f5e998de2307215f50eda0b1"} Jun 08 18:56:26.636553 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:26.636380 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:26.636553 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:26.636399 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" event={"ID":"836ab0f5-0220-42dd-9f42-c46c30a7f9ca","Type":"ContainerStarted","Data":"fe001edfbd779229f27adca7f7d762f5fa675aef41ddfaf16b72faf9093fc21e"} Jun 08 18:56:26.653981 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:26.653925 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" podStartSLOduration=1.6539090170000001 podStartE2EDuration="1.653909017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:56:26.65329018 +0000 UTC m=+2460.814126835" watchObservedRunningTime="2026-06-08 18:56:26.653909017 +0000 UTC m=+2460.814745674" Jun 08 18:56:27.152606 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:27.152576 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8cs4k_c4835af7-f4d0-4820-ba05-41ac0bcf4b66/node-ca/0.log" Jun 08 18:56:28.128606 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:28.128578 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795lsj7w_f04b0cf9-3011-4541-92bf-49510e2699d0/istio-proxy/0.log" Jun 08 18:56:28.348684 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:28.348649 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p5fnj_a23ba43d-8f9b-47f1-a68c-b175c25bdc95/payload-pre-processing/0.log" Jun 08 18:56:28.404995 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:28.404912 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9m7hr_74581d47-cce4-49b4-a980-0cc0e48c05ce/payload-processing/0.log" Jun 08 18:56:29.048045 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:29.048015 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8pb2c_1bb37a57-1c66-4ca1-8bc3-278f5e0c5f93/serve-healthcheck-canary/0.log" Jun 08 18:56:29.732528 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:29.732502 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xbf77_0171afcc-f8cd-4930-8d23-04e7825b2721/kube-rbac-proxy/0.log" Jun 08 18:56:29.754310 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:29.754281 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xbf77_0171afcc-f8cd-4930-8d23-04e7825b2721/exporter/0.log" Jun 08 18:56:29.775368 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:29.775347 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xbf77_0171afcc-f8cd-4930-8d23-04e7825b2721/extractor/0.log" Jun 08 18:56:31.950691 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:31.950666 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-84b585c54b-5vkjs_b8c3624d-0ecd-407e-ba87-c45384ad36be/nginx/0.log" Jun 08 18:56:32.108412 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:32.108380 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-77d5447464-822qr_f9bca354-5115-46fb-b6de-3981ba703534/maas-api/0.log" Jun 08 18:56:32.191721 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:32.191693 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cb9f9847-bz9jn_e7fed784-1336-4a75-be7f-0b5f17043b2e/manager/0.log" Jun 08 18:56:32.477551 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:32.477515 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4695b4ff-lxvrm_6f9e5797-8a69-4c8b-af51-6271c1ae2ea3/manager/0.log" Jun 08 18:56:32.498587 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:32.498562 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b4695b4ff-lxvrm_6f9e5797-8a69-4c8b-af51-6271c1ae2ea3/copy-manifests/0.log" Jun 08 18:56:32.651484 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:32.651463 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2nmtb/perf-node-gather-daemonset-sn2vj" Jun 08 18:56:34.136887 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:34.136863 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5795b6f5cf-q5mcr_768f7c03-86e5-4c54-8370-d6a0a87b0cde/manager/0.log" Jun 08 18:56:37.844604 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:37.844577 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795lsj7w_f04b0cf9-3011-4541-92bf-49510e2699d0/istio-proxy/0.log" Jun 08 18:56:38.012937 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:38.012908 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p5fnj_a23ba43d-8f9b-47f1-a68c-b175c25bdc95/payload-pre-processing/0.log" Jun 08 18:56:38.058370 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:38.058339 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9m7hr_74581d47-cce4-49b4-a980-0cc0e48c05ce/payload-processing/0.log" Jun 08 18:56:39.097379 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:39.097347 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795lsj7w_f04b0cf9-3011-4541-92bf-49510e2699d0/istio-proxy/0.log" Jun 08 18:56:39.274048 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:39.274020 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p5fnj_a23ba43d-8f9b-47f1-a68c-b175c25bdc95/payload-pre-processing/0.log" Jun 08 18:56:39.326209 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:39.326178 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9m7hr_74581d47-cce4-49b4-a980-0cc0e48c05ce/payload-processing/0.log" Jun 08 18:56:39.594000 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:39.593969 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-vcw56_ad947046-f104-42fc-83f7-b6bbd26fd565/migrator/0.log" Jun 08 18:56:39.622806 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:39.622781 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-vcw56_ad947046-f104-42fc-83f7-b6bbd26fd565/graceful-termination/0.log" Jun 08 18:56:40.056186 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:40.056104 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-v7298_921316a0-b53b-447d-a7a6-e9c0d8b4d208/kube-storage-version-migrator-operator/1.log" Jun 08 18:56:40.056885 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:40.056865 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-v7298_921316a0-b53b-447d-a7a6-e9c0d8b4d208/kube-storage-version-migrator-operator/0.log" Jun 08 18:56:41.004193 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.004150 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795lsj7w_f04b0cf9-3011-4541-92bf-49510e2699d0/istio-proxy/0.log" Jun 08 18:56:41.174193 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.174174 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p5fnj_a23ba43d-8f9b-47f1-a68c-b175c25bdc95/payload-pre-processing/0.log" Jun 08 18:56:41.204551 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.204527 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26l7d_d1540b0a-c591-4239-8ac2-41fd39c3eae3/kube-multus-additional-cni-plugins/0.log" Jun 08 18:56:41.223760 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.223732 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9m7hr_74581d47-cce4-49b4-a980-0cc0e48c05ce/payload-processing/0.log" Jun 08 18:56:41.225764 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.225743 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26l7d_d1540b0a-c591-4239-8ac2-41fd39c3eae3/egress-router-binary-copy/0.log" Jun 08 18:56:41.245568 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.245545 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26l7d_d1540b0a-c591-4239-8ac2-41fd39c3eae3/cni-plugins/0.log" Jun 08 18:56:41.266786 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.266732 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26l7d_d1540b0a-c591-4239-8ac2-41fd39c3eae3/bond-cni-plugin/0.log" Jun 08 18:56:41.289378 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.289354 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26l7d_d1540b0a-c591-4239-8ac2-41fd39c3eae3/routeoverride-cni/0.log" Jun 08 18:56:41.309703 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.309685 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26l7d_d1540b0a-c591-4239-8ac2-41fd39c3eae3/whereabouts-cni-bincopy/0.log" Jun 08 18:56:41.330353 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.330333 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-26l7d_d1540b0a-c591-4239-8ac2-41fd39c3eae3/whereabouts-cni/0.log" Jun 08 18:56:41.753921 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.753886 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vslwg_f4232b10-00c4-4f03-b7fa-d602b432a60c/kube-multus/0.log" Jun 08 18:56:41.820430 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.820406 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nk59b_e5c81386-a373-411b-9eb1-205634bdae58/network-metrics-daemon/0.log" Jun 08 18:56:41.842131 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:41.842110 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nk59b_e5c81386-a373-411b-9eb1-205634bdae58/kube-rbac-proxy/0.log" Jun 08 18:56:42.779923 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:42.779898 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-controller/0.log" Jun 08 18:56:42.804017 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:42.803991 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/0.log" Jun 08 18:56:42.804150 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:42.804035 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/ovn-acl-logging/1.log" Jun 08 18:56:42.825734 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:42.825709 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/kube-rbac-proxy-node/0.log" Jun 08 18:56:42.850830 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:42.850799 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/kube-rbac-proxy-ovn-metrics/0.log" Jun 08 18:56:42.871912 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:42.871889 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/northd/0.log" Jun 08 18:56:42.893740 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:42.893719 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/nbdb/0.log" Jun 08 18:56:42.915619 ip-10-0-131-227 kubenswrapper[2560]: I0608 18:56:42.915593 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4xn9_3985b2b9-e6b3-4430-ae83-217a9afde35f/sbdb/0.log"