Jun 02 19:21:09.810878 ip-10-0-138-231 systemd[1]: Starting Kubernetes Kubelet... Jun 02 19:21:10.162626 ip-10-0-138-231 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:21:10.162626 ip-10-0-138-231 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 19:21:10.162626 ip-10-0-138-231 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:21:10.162626 ip-10-0-138-231 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 19:21:10.162626 ip-10-0-138-231 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:21:10.163975 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.163873 2552 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 19:21:10.166726 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166708 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:21:10.166726 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166727 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166732 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166735 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166739 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166743 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166745 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166748 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166752 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166754 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166757 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166760 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166763 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166766 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166770 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166773 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166775 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166780 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166783 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166786 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166789 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:21:10.166800 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166791 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166794 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166796 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166799 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166802 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166805 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166808 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166810 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166813 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166816 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166819 2552 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166822 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166824 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166827 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166829 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166832 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166834 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166836 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166839 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166841 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:21:10.167283 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166844 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166846 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166849 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166851 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166854 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166856 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166860 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166863 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166865 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166868 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166870 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166873 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166875 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166878 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166881 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166883 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166887 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166890 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166893 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166895 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166898 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:21:10.167829 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166900 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166903 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166905 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166908 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166910 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166913 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166915 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166918 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166920 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166923 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166926 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166928 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166931 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166933 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166936 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166938 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166940 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166944 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166947 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:21:10.168362 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166950 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166952 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166955 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166958 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.166961 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167371 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167376 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167379 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167382 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167385 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167387 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167390 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167394 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167398 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167402 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167405 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167408 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167411 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167414 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:21:10.168862 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167417 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167420 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167422 2552 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167425 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167427 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167430 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167433 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167436 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167438 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167441 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167444 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167447 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167450 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167454 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167457 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167459 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167462 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167465 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167469 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167472 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:21:10.169407 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167474 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167477 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167480 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167484 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167486 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167503 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167506 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167509 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167511 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167514 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167517 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167519 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167522 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167525 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167527 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167530 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167537 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167540 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167542 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:21:10.169955 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167545 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167548 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167550 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167553 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167555 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167558 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167562 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167565 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167568 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167571 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167573 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167577 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167580 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167582 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167585 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167587 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167590 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167593 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167595 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167598 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167601 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:21:10.170457 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167604 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167606 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167609 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167612 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167614 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167618 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167620 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167623 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167626 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167629 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167631 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.167634 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168874 2552 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168887 2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168896 2552 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168900 2552 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168905 2552 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168909 2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168918 2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168923 2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168926 2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 19:21:10.170990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168929 2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168933 2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168937 2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168940 2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168943 2552 flags.go:64] FLAG: --cgroup-root="" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168946 2552 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168949 2552 flags.go:64] FLAG: --client-ca-file="" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168952 2552 flags.go:64] FLAG: --cloud-config="" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168955 2552 flags.go:64] FLAG: --cloud-provider="external" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168958 2552 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168962 2552 flags.go:64] FLAG: --cluster-domain="" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168965 2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168968 2552 flags.go:64] FLAG: --config-dir="" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168971 2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168975 2552 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168979 2552 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168982 2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168985 2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168988 2552 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168994 2552 flags.go:64] FLAG: --contention-profiling="false" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.168997 2552 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169000 2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169003 2552 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169006 2552 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169010 2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 19:21:10.171533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169013 2552 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169016 2552 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169019 2552 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169023 2552 flags.go:64] FLAG: --enable-server="true" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169026 2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169040 2552 flags.go:64] FLAG: --event-burst="100" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169043 2552 flags.go:64] FLAG: --event-qps="50" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169047 2552 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169051 2552 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169054 2552 flags.go:64] FLAG: --eviction-hard="" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169058 2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169061 2552 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169064 2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169067 2552 flags.go:64] FLAG: --eviction-soft="" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169070 2552 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169073 2552 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169076 2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169079 2552 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169082 2552 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169085 2552 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169088 2552 flags.go:64] FLAG: --feature-gates="" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169092 2552 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169095 2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169098 2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169102 2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169105 2552 flags.go:64] FLAG: --healthz-port="10248" Jun 02 19:21:10.172146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169109 2552 flags.go:64] FLAG: --help="false" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169112 2552 flags.go:64] FLAG: --hostname-override="ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169115 2552 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169119 2552 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169122 2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169125 2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169128 2552 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169131 2552 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169134 2552 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169137 2552 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169140 2552 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169143 2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169146 2552 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169149 2552 flags.go:64] FLAG: --kube-reserved="" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169153 2552 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169156 2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169159 2552 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169162 2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169165 2552 flags.go:64] FLAG: --lock-file="" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169168 2552 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169171 2552 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169174 2552 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169180 2552 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169183 2552 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 19:21:10.172821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169186 2552 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169189 2552 flags.go:64] FLAG: --logging-format="text" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169191 2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169195 2552 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169197 2552 flags.go:64] FLAG: --manifest-url="" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169200 2552 flags.go:64] FLAG: --manifest-url-header="" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169205 2552 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169208 2552 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169217 2552 flags.go:64] FLAG: --max-pods="110" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169220 2552 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169223 2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169226 2552 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169229 2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169232 2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169235 2552 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169238 2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169247 2552 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169250 2552 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169253 2552 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169256 2552 flags.go:64] FLAG: --pod-cidr="" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169259 2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169266 2552 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169269 2552 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169273 2552 flags.go:64] FLAG: --pods-per-core="0" Jun 02 19:21:10.173406 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169277 2552 flags.go:64] FLAG: --port="10250" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169280 2552 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169283 2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-091552d74e9a9c474" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169286 2552 flags.go:64] FLAG: --qos-reserved="" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169289 2552 flags.go:64] FLAG: --read-only-port="10255" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169292 2552 flags.go:64] FLAG: --register-node="true" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169295 2552 flags.go:64] FLAG: --register-schedulable="true" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169298 2552 flags.go:64] FLAG: --register-with-taints="" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169301 2552 flags.go:64] FLAG: --registry-burst="10" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169304 2552 flags.go:64] FLAG: --registry-qps="5" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169307 2552 flags.go:64] FLAG: --reserved-cpus="" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169310 2552 flags.go:64] FLAG: --reserved-memory="" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169314 2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169317 2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169320 2552 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169323 2552 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169328 2552 flags.go:64] FLAG: --runonce="false" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169331 2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169334 2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169337 2552 flags.go:64] FLAG: --seccomp-default="false" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169340 2552 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169343 2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169346 2552 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169349 2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169352 2552 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169355 2552 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 19:21:10.174017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169358 2552 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169361 2552 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169364 2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169367 2552 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169371 2552 flags.go:64] FLAG: --system-cgroups="" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169374 2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169380 2552 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169383 2552 flags.go:64] FLAG: --tls-cert-file="" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169386 2552 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169393 2552 flags.go:64] FLAG: --tls-min-version="" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169396 2552 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169399 2552 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169402 2552 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169404 2552 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169408 2552 flags.go:64] FLAG: --v="2" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169413 2552 flags.go:64] FLAG: --version="false" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169417 2552 flags.go:64] FLAG: --vmodule="" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169422 2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.169425 2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169555 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169560 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169563 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169567 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169570 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:21:10.174661 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169573 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169576 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169579 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169581 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169584 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169587 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169589 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169592 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169594 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169597 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169600 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169602 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169606 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169617 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169620 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169623 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169625 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169628 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169631 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169633 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:21:10.175233 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169636 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169638 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169641 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169644 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169646 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169649 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169651 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169654 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169657 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169659 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169662 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169664 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169667 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169669 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169672 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169675 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169677 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169680 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169682 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169685 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:21:10.175945 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169687 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169690 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169692 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169695 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169698 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169701 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169711 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169714 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169717 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169720 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169723 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169725 2552 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169728 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169731 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169734 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169736 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169739 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169742 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169746 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:21:10.176892 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169749 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169752 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169755 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169758 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169761 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169764 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169767 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169769 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169772 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169781 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169784 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169786 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169789 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169791 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169794 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169796 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169801 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169803 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169806 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169809 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:21:10.177466 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169815 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:21:10.178287 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.169817 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:21:10.178287 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.170344 2552 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:21:10.179347 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.179323 2552 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 19:21:10.179347 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.179347 2552 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179401 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179408 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179411 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179415 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179418 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179421 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179423 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179426 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179429 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179432 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179434 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179438 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179440 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179443 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:21:10.179441 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179446 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179449 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179452 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179456 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179459 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179461 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179464 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179467 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179470 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179474 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179479 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179482 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179485 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179489 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179508 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179511 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179514 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179516 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179519 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179522 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:21:10.179846 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179524 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179527 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179529 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179532 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179534 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179537 2552 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179539 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179542 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179545 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179550 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179553 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179555 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179558 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179561 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179564 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179567 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179570 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179573 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179576 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179579 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:21:10.180343 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179581 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179584 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179587 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179589 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179592 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179594 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179597 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179600 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179602 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179605 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179608 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179611 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179615 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179618 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179621 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179623 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179626 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179629 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179631 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179634 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:21:10.180864 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179636 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179639 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179642 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179645 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179647 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179650 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179653 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179656 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179659 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179661 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179664 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179667 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.179672 2552 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179774 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179779 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179782 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:21:10.181357 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179785 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179787 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179790 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179793 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179795 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179798 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179800 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179803 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179806 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179808 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179811 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179814 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179816 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179820 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179822 2552 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179825 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179827 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179830 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179832 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179836 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:21:10.181779 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179839 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179841 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179844 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179848 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179851 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179853 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179856 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179859 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179862 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179864 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179867 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179869 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179872 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179875 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179878 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179881 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179883 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179886 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179888 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179891 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:21:10.182267 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179894 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179896 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179899 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179903 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179906 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179909 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179912 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179914 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179917 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179920 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179923 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179926 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179931 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179934 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179938 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179941 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179944 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179946 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179949 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:21:10.182765 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179951 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179954 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179957 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179959 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179962 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179965 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179967 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179970 2552 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179972 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179975 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179978 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179980 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179983 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179985 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179988 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179990 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179993 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179995 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.179998 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.180000 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:21:10.183237 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.180003 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.180006 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.180008 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:10.180010 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.180016 2552 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.180636 2552 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.182792 2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.183582 2552 server.go:1019] "Starting client certificate rotation" Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.183679 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 19:21:10.183737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.183718 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 19:21:10.203299 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.203276 2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 19:21:10.205813 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.205790 2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 19:21:10.217295 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.217276 2552 log.go:25] "Validated CRI v1 runtime API" Jun 02 19:21:10.222696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.222679 2552 log.go:25] "Validated CRI v1 image API" Jun 02 19:21:10.223878 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.223860 2552 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 19:21:10.227576 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.227550 2552 fs.go:135] Filesystem UUIDs: map[431d4704-20c1-485a-b844-29a314f6e126:/dev/nvme0n1p3 5e0d44c5-7228-455f-96d4-1dd27d9dceaa:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 02 19:21:10.227654 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.227575 2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 19:21:10.232284 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.232262 2552 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 19:21:10.233626 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.233516 2552 manager.go:217] Machine: {Timestamp:2026-06-02 19:21:10.232999302 +0000 UTC m=+0.322867526 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499994 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20720aabdcc5c5171e9db401d9d78c SystemUUID:ec20720a-abdc-c5c5-171e-9db401d9d78c BootID:e26232a7-ee6c-4574-ad20-3d429db248d3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:68:4f:6d:ed:69 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:68:4f:6d:ed:69 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:4f:6e:26:6d:d6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 19:21:10.234209 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.234195 2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 19:21:10.234324 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.234310 2552 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 19:21:10.235291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.235265 2552 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 19:21:10.235429 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.235293 2552 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-231.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 19:21:10.235473 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.235440 2552 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 19:21:10.235473 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.235448 2552 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 19:21:10.235473 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.235456 2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 19:21:10.236059 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.236049 2552 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 19:21:10.237325 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.237315 2552 state_mem.go:36] "Initialized new in-memory state store" Jun 02 19:21:10.237446 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.237436 2552 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 19:21:10.239899 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.239889 2552 kubelet.go:491] "Attempting to sync node with API server" Jun 02 19:21:10.239939 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.239905 2552 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 19:21:10.239939 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.239925 2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 19:21:10.239939 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.239939 2552 kubelet.go:397] "Adding apiserver pod source" Jun 02 19:21:10.240016 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.239949 2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 19:21:10.240905 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.240892 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 19:21:10.240967 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.240911 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 19:21:10.243774 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.243756 2552 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 19:21:10.245375 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.245362 2552 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 19:21:10.247258 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247246 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 19:21:10.247295 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247265 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 19:21:10.247295 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247272 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 19:21:10.247295 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247278 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 19:21:10.247295 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247287 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 19:21:10.247295 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247294 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 19:21:10.247441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247299 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 19:21:10.247441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247305 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 19:21:10.247441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247314 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 19:21:10.247441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247320 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 19:21:10.247441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247328 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 19:21:10.247441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.247334 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 19:21:10.248578 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.248568 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 19:21:10.248578 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.248579 2552 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 19:21:10.251833 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.251819 2552 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 19:21:10.251896 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.251856 2552 server.go:1295] "Started kubelet" Jun 02 19:21:10.251945 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.251912 2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 19:21:10.252084 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.252039 2552 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 19:21:10.252134 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.252114 2552 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 19:21:10.252899 ip-10-0-138-231 systemd[1]: Started Kubernetes Kubelet. Jun 02 19:21:10.253090 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.253039 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 19:21:10.253180 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.253138 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-231.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 19:21:10.253180 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.253148 2552 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 19:21:10.253284 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.253256 2552 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-231.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 19:21:10.255130 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.255115 2552 server.go:317] "Adding debug handlers to kubelet server" Jun 02 19:21:10.255474 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.255446 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8kptx" Jun 02 19:21:10.257915 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.257189 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-231.ec2.internal.18b559e5937ea271 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-231.ec2.internal,UID:ip-10-0-138-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-231.ec2.internal,},FirstTimestamp:2026-06-02 19:21:10.251831921 +0000 UTC m=+0.341700148,LastTimestamp:2026-06-02 19:21:10.251831921 +0000 UTC m=+0.341700148,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-231.ec2.internal,}" Jun 02 19:21:10.258060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.258040 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 19:21:10.258403 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.258390 2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 19:21:10.258556 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.258539 2552 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 19:21:10.258644 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.258635 2552 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 19:21:10.258815 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.258799 2552 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 19:21:10.258888 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.258837 2552 reconstruct.go:97] "Volume reconstruction finished" Jun 02 19:21:10.258888 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.258847 2552 reconciler.go:26] "Reconciler: start to sync state" Jun 02 19:21:10.259394 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.259376 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:10.261110 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.259444 2552 factory.go:153] Registering CRI-O factory Jun 02 19:21:10.261206 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.261117 2552 factory.go:223] Registration of the crio container factory successfully Jun 02 19:21:10.261206 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.261174 2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 19:21:10.261206 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.261184 2552 factory.go:55] Registering systemd factory Jun 02 19:21:10.261206 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.261192 2552 factory.go:223] Registration of the systemd container factory successfully Jun 02 19:21:10.261206 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.261210 2552 factory.go:103] Registering Raw factory Jun 02 19:21:10.261433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.261224 2552 manager.go:1196] Started watching for new ooms in manager Jun 02 19:21:10.261433 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.259968 2552 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 19:21:10.261433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.261259 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8kptx" Jun 02 19:21:10.261897 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.261881 2552 manager.go:319] Starting recovery of all containers Jun 02 19:21:10.266935 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.266142 2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-231.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 19:21:10.266935 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.266293 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 19:21:10.273988 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.273825 2552 manager.go:324] Recovery completed Jun 02 19:21:10.278441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.278425 2552 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:21:10.280530 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.280514 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:21:10.280635 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.280554 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:21:10.280635 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.280569 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:21:10.281136 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.281119 2552 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 19:21:10.281136 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.281134 2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 19:21:10.281233 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.281152 2552 state_mem.go:36] "Initialized new in-memory state store" Jun 02 19:21:10.283477 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.283464 2552 policy_none.go:49] "None policy: Start" Jun 02 19:21:10.283538 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.283482 2552 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 19:21:10.283538 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.283507 2552 state_mem.go:35] "Initializing new in-memory state store" Jun 02 19:21:10.321188 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.321168 2552 manager.go:341] "Starting Device Plugin manager" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.321228 2552 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.321243 2552 server.go:85] "Starting device plugin registration server" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.321584 2552 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.321602 2552 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.321688 2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.321777 2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.321785 2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.322376 2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 19:21:10.331051 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.322413 2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:10.387590 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.387544 2552 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 19:21:10.388811 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.388794 2552 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 19:21:10.388911 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.388831 2552 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 19:21:10.388911 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.388857 2552 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 19:21:10.388911 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.388867 2552 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 19:21:10.389039 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.388913 2552 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 19:21:10.391669 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.391646 2552 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:21:10.421787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.421713 2552 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:21:10.422854 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.422838 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:21:10.422935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.422873 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:21:10.422935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.422886 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:21:10.422935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.422913 2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.428961 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.428944 2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.429001 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.428971 2552 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-231.ec2.internal\": node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:10.451922 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.451886 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:10.489166 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.489133 2552 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal"] Jun 02 19:21:10.489325 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.489220 2552 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:21:10.490179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.490163 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:21:10.490275 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.490195 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:21:10.490275 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.490209 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:21:10.491483 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.491467 2552 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:21:10.491696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.491659 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.491696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.491691 2552 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:21:10.492236 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.492217 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:21:10.492334 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.492217 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:21:10.492334 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.492250 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:21:10.492334 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.492261 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:21:10.492334 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.492270 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:21:10.492334 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.492285 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:21:10.493237 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.493223 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.493295 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.493248 2552 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:21:10.493937 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.493924 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:21:10.493989 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.493946 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:21:10.493989 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.493958 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:21:10.510076 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.510057 2552 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-231.ec2.internal\" not found" node="ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.513946 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.513928 2552 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-231.ec2.internal\" not found" node="ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.552363 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.552327 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:10.561153 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.561129 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e335eccfebc403b8053a4893ae8bbe1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal\" (UID: \"2e335eccfebc403b8053a4893ae8bbe1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.561300 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.561164 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03340f265015a5ce3cd39b70624e15fb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-231.ec2.internal\" (UID: \"03340f265015a5ce3cd39b70624e15fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.561300 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.561191 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e335eccfebc403b8053a4893ae8bbe1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal\" (UID: \"2e335eccfebc403b8053a4893ae8bbe1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.653139 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.653109 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:10.661483 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.661458 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03340f265015a5ce3cd39b70624e15fb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-231.ec2.internal\" (UID: \"03340f265015a5ce3cd39b70624e15fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.661614 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.661512 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e335eccfebc403b8053a4893ae8bbe1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal\" (UID: \"2e335eccfebc403b8053a4893ae8bbe1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.661614 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.661538 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e335eccfebc403b8053a4893ae8bbe1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal\" (UID: \"2e335eccfebc403b8053a4893ae8bbe1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.661614 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.661547 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03340f265015a5ce3cd39b70624e15fb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-231.ec2.internal\" (UID: \"03340f265015a5ce3cd39b70624e15fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.661614 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.661597 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e335eccfebc403b8053a4893ae8bbe1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal\" (UID: \"2e335eccfebc403b8053a4893ae8bbe1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.661768 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.661649 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e335eccfebc403b8053a4893ae8bbe1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal\" (UID: \"2e335eccfebc403b8053a4893ae8bbe1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.753873 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.753782 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:10.812313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.812277 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.817794 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:10.817778 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" Jun 02 19:21:10.854482 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.854456 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:10.954947 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:10.954919 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:11.055440 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:11.055380 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:11.155889 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:11.155860 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:11.183346 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.183317 2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 19:21:11.184043 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.183505 2552 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 19:21:11.224167 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.224136 2552 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:21:11.255981 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:11.255945 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:11.259175 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.259145 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 19:21:11.263403 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.263369 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 19:16:10 +0000 UTC" deadline="2028-03-19 14:31:15.668212656 +0000 UTC" Jun 02 19:21:11.263403 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.263401 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15739h10m4.404815406s" Jun 02 19:21:11.272257 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.272234 2552 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 19:21:11.274476 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:11.274445 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03340f265015a5ce3cd39b70624e15fb.slice/crio-ad495f975812e80ab4ae912e5a46aac92548af9e73f842e2eb2936ceb63ea69e WatchSource:0}: Error finding container ad495f975812e80ab4ae912e5a46aac92548af9e73f842e2eb2936ceb63ea69e: Status 404 returned error can't find the container with id ad495f975812e80ab4ae912e5a46aac92548af9e73f842e2eb2936ceb63ea69e Jun 02 19:21:11.274795 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:11.274772 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e335eccfebc403b8053a4893ae8bbe1.slice/crio-6c104591cd557525e6f675267fca8290155f04f0389dae1e886beb275d9702c0 WatchSource:0}: Error finding container 6c104591cd557525e6f675267fca8290155f04f0389dae1e886beb275d9702c0: Status 404 returned error can't find the container with id 6c104591cd557525e6f675267fca8290155f04f0389dae1e886beb275d9702c0 Jun 02 19:21:11.278019 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.278005 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:21:11.308153 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.308079 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-krd52" Jun 02 19:21:11.316205 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.316184 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-krd52" Jun 02 19:21:11.356723 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:11.356684 2552 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-231.ec2.internal\" not found" Jun 02 19:21:11.383622 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.383593 2552 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:21:11.392388 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.392341 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" event={"ID":"2e335eccfebc403b8053a4893ae8bbe1","Type":"ContainerStarted","Data":"6c104591cd557525e6f675267fca8290155f04f0389dae1e886beb275d9702c0"} Jun 02 19:21:11.393291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.393266 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" event={"ID":"03340f265015a5ce3cd39b70624e15fb","Type":"ContainerStarted","Data":"ad495f975812e80ab4ae912e5a46aac92548af9e73f842e2eb2936ceb63ea69e"} Jun 02 19:21:11.459438 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.459411 2552 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" Jun 02 19:21:11.470982 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.470959 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 19:21:11.471700 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.471688 2552 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" Jun 02 19:21:11.482354 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.482336 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 19:21:11.609971 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:11.609879 2552 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:21:12.164559 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.164471 2552 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:21:12.240897 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.240858 2552 apiserver.go:52] "Watching apiserver" Jun 02 19:21:12.248586 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.248548 2552 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 19:21:12.250565 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.250536 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal","openshift-multus/multus-wxwcg","openshift-network-operator/iptables-alerter-wrxx5","openshift-ovn-kubernetes/ovnkube-node-4xppd","kube-system/konnectivity-agent-cxvvr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw","openshift-cluster-node-tuning-operator/tuned-fnhpv","openshift-dns/node-resolver-mc8fl","openshift-image-registry/node-ca-26fj5","openshift-multus/multus-additional-cni-plugins-pxw6v","openshift-multus/network-metrics-daemon-sgj2b","openshift-network-diagnostics/network-check-target-kscb7"] Jun 02 19:21:12.252939 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.252919 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.254064 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.254036 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.254177 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.254155 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.255526 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.255505 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mgsss\"" Jun 02 19:21:12.255615 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.255530 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 19:21:12.255615 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.255539 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 19:21:12.255615 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.255557 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 19:21:12.256589 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.256434 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 19:21:12.256589 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.256446 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:21:12.256589 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.256548 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 19:21:12.256796 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.256756 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.256849 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.256795 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 19:21:12.256896 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.256869 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 19:21:12.257082 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.257064 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 19:21:12.257380 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.257362 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s8m8r\"" Jun 02 19:21:12.257657 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.257639 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-knfhz\"" Jun 02 19:21:12.257743 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.257669 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:12.257847 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.257831 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.258961 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.258939 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 19:21:12.259705 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.259686 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 19:21:12.260060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.260040 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 19:21:12.260146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.260043 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 19:21:12.260146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.260084 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 19:21:12.260146 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.260127 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 19:21:12.260295 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.260074 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 19:21:12.261598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.260802 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lgqg4\"" Jun 02 19:21:12.261598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.260869 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q54x9\"" Jun 02 19:21:12.261598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.261018 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 19:21:12.261598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.261450 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:21:12.261834 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.261707 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.262150 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.261906 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7w86s\"" Jun 02 19:21:12.262150 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.261978 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 19:21:12.262302 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.262282 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 19:21:12.263870 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.263839 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.264026 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.264009 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.265226 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.265105 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 19:21:12.265226 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.265109 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kbx86\"" Jun 02 19:21:12.265226 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.265166 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 19:21:12.265980 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.265961 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:12.266071 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.266048 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:12.266400 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.266378 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 19:21:12.266488 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.266446 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pmkp8\"" Jun 02 19:21:12.266703 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.266686 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 19:21:12.266791 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.266704 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 19:21:12.266914 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.266900 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fpt5d\"" Jun 02 19:21:12.266973 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.266965 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 19:21:12.267287 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.267272 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:12.267367 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.267329 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:12.267504 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.267475 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 19:21:12.270670 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270650 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-cni-multus\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.270758 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270677 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-host\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.270758 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270700 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-tuned\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.270758 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270725 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ec25af5a-4ef3-479c-8c81-0251ed117c8c-iptables-alerter-script\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.270758 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270749 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-slash\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.270962 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270774 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.270962 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270798 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/859620d1-906d-4775-bf20-d888549445d4-hosts-file\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.270962 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270820 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-etc-kubernetes\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.270962 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270842 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-socket-dir-parent\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.270962 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270867 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-cni-bin\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.270962 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270904 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec25af5a-4ef3-479c-8c81-0251ed117c8c-host-slash\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.270962 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270958 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-kubelet\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.270993 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4kkc\" (UniqueName: \"kubernetes.io/projected/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-kube-api-access-z4kkc\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271020 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-etc-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271051 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271075 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-modprobe-d\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271098 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98907b61-a541-4b22-8cd6-0f31f6baea96-serviceca\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271116 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-cni-netd\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271129 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a375452-9057-4bd7-9337-422d42b662e0-agent-certs\") pod \"konnectivity-agent-cxvvr\" (UID: \"8a375452-9057-4bd7-9337-422d42b662e0\") " pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271143 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a375452-9057-4bd7-9337-422d42b662e0-konnectivity-ca\") pod \"konnectivity-agent-cxvvr\" (UID: \"8a375452-9057-4bd7-9337-422d42b662e0\") " pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271171 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271191 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859620d1-906d-4775-bf20-d888549445d4-tmp-dir\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271207 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98907b61-a541-4b22-8cd6-0f31f6baea96-host\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271231 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-daemon-config\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.271292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271248 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-device-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271304 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-sys-fs\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271340 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-cnibin\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271384 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-lib-modules\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271406 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-tmp\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271431 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b7e9986-2590-42d6-8fba-e49e780ebc72-cni-binary-copy\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271452 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysctl-d\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271475 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysctl-conf\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271513 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j796\" (UniqueName: \"kubernetes.io/projected/f8636918-e4cf-4852-b6e7-640039dca7b8-kube-api-access-2j796\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271551 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271574 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-k8s-cni-cncf-io\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271596 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-env-overrides\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271630 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-cni-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271653 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-conf-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271677 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln44t\" (UniqueName: \"kubernetes.io/projected/8b7e9986-2590-42d6-8fba-e49e780ebc72-kube-api-access-ln44t\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271699 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-kubernetes\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271721 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-systemd\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.271941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271743 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-664l4\" (UniqueName: \"kubernetes.io/projected/859620d1-906d-4775-bf20-d888549445d4-kube-api-access-664l4\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271758 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-socket-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271774 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-system-cni-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271803 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-kubelet\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271836 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-systemd\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271874 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-ovn\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271907 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6lz\" (UniqueName: \"kubernetes.io/projected/51bf5173-3887-417e-a765-f9f7f0a2e099-kube-api-access-ql6lz\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271932 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-os-release\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271952 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-hostroot\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271966 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-sys\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.271985 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-run-netns\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272007 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-var-lib-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272028 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-log-socket\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272050 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-registration-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272072 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-systemd-units\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272127 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-cni-bin\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272164 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-multus-certs\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.272838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272242 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysconfig\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272312 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-node-log\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272379 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8636918-e4cf-4852-b6e7-640039dca7b8-ovn-node-metrics-cert\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272418 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfmn5\" (UniqueName: \"kubernetes.io/projected/ec25af5a-4ef3-479c-8c81-0251ed117c8c-kube-api-access-mfmn5\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272459 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-var-lib-kubelet\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272479 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-ovnkube-config\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272516 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-ovnkube-script-lib\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272545 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-netns\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272598 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-run\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272630 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwxng\" (UniqueName: \"kubernetes.io/projected/98907b61-a541-4b22-8cd6-0f31f6baea96-kube-api-access-kwxng\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.273612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.272657 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.316917 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.316881 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 19:16:11 +0000 UTC" deadline="2028-01-05 20:25:16.959209088 +0000 UTC" Jun 02 19:21:12.316917 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.316915 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13969h4m4.64229807s" Jun 02 19:21:12.360459 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.360372 2552 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 19:21:12.373225 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373176 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mfmn5\" (UniqueName: \"kubernetes.io/projected/ec25af5a-4ef3-479c-8c81-0251ed117c8c-kube-api-access-mfmn5\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.373378 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373232 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-var-lib-kubelet\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.373378 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373266 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-ovnkube-config\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.373378 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373329 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-ovnkube-script-lib\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.373378 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373373 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmkln\" (UniqueName: \"kubernetes.io/projected/386d5a57-cdc7-4d70-b3af-c06ea739c213-kube-api-access-wmkln\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:12.373583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373409 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-netns\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.373583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373443 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-run\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.373583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373472 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kwxng\" (UniqueName: \"kubernetes.io/projected/98907b61-a541-4b22-8cd6-0f31f6baea96-kube-api-access-kwxng\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.373583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373541 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.373754 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373689 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-cni-binary-copy\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.373795 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373778 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.373841 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373814 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-cni-multus\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.373929 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373899 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-host\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.373975 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373933 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-tuned\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.373975 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373967 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ec25af5a-4ef3-479c-8c81-0251ed117c8c-iptables-alerter-script\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.374064 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.373996 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-slash\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.374064 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374028 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.374158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374069 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/859620d1-906d-4775-bf20-d888549445d4-hosts-file\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.374158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374062 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-var-lib-kubelet\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.374158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374149 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/859620d1-906d-4775-bf20-d888549445d4-hosts-file\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.374294 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374168 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-run\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.374294 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374194 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-netns\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.374390 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374344 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374467 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-cni-multus\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374563 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-host\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374661 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-system-cni-dir\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374709 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-etc-kubernetes\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374775 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-socket-dir-parent\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374755 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-ovnkube-config\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374822 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-cni-bin\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374849 2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374864 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec25af5a-4ef3-479c-8c81-0251ed117c8c-host-slash\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374903 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374954 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-kubelet\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.374991 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z4kkc\" (UniqueName: \"kubernetes.io/projected/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-kube-api-access-z4kkc\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375070 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-etc-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375110 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375176 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375209 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-modprobe-d\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375235 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98907b61-a541-4b22-8cd6-0f31f6baea96-serviceca\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.375750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375268 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-cni-netd\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375311 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a375452-9057-4bd7-9337-422d42b662e0-agent-certs\") pod \"konnectivity-agent-cxvvr\" (UID: \"8a375452-9057-4bd7-9337-422d42b662e0\") " pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375339 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a375452-9057-4bd7-9337-422d42b662e0-konnectivity-ca\") pod \"konnectivity-agent-cxvvr\" (UID: \"8a375452-9057-4bd7-9337-422d42b662e0\") " pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375369 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375472 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859620d1-906d-4775-bf20-d888549445d4-tmp-dir\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375535 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98907b61-a541-4b22-8cd6-0f31f6baea96-host\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375595 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-daemon-config\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375627 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-device-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375655 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-sys-fs\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375678 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-cnibin\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.375924 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-modprobe-d\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376173 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376343 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859620d1-906d-4775-bf20-d888549445d4-tmp-dir\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376379 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-cnibin\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376450 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a375452-9057-4bd7-9337-422d42b662e0-konnectivity-ca\") pod \"konnectivity-agent-cxvvr\" (UID: \"8a375452-9057-4bd7-9337-422d42b662e0\") " pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376573 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-etc-kubernetes\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376520 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-socket-dir-parent\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.376648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376636 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376571 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-cni-bin\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376666 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-device-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376679 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-sys-fs\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376730 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-etc-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.376850 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-var-lib-kubelet\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377025 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-ovnkube-script-lib\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377063 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98907b61-a541-4b22-8cd6-0f31f6baea96-serviceca\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377100 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98907b61-a541-4b22-8cd6-0f31f6baea96-host\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377135 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ec25af5a-4ef3-479c-8c81-0251ed117c8c-iptables-alerter-script\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377157 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-slash\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377176 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-lib-modules\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377223 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-tmp\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377259 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b7e9986-2590-42d6-8fba-e49e780ebc72-cni-binary-copy\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377284 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysctl-d\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377324 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-cni-netd\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377332 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysctl-conf\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377387 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2j796\" (UniqueName: \"kubernetes.io/projected/f8636918-e4cf-4852-b6e7-640039dca7b8-kube-api-access-2j796\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.377410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377405 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec25af5a-4ef3-479c-8c81-0251ed117c8c-host-slash\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377426 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377439 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysctl-conf\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377512 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377630 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377769 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-lib-modules\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377829 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-k8s-cni-cncf-io\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377880 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-env-overrides\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377906 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysctl-d\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377913 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-k8s-cni-cncf-io\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377912 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-cnibin\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377831 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.377960 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-cni-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.378224 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378023 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-conf-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.378822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378610 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-cni-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.378822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378641 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ln44t\" (UniqueName: \"kubernetes.io/projected/8b7e9986-2590-42d6-8fba-e49e780ebc72-kube-api-access-ln44t\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.378822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378683 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-kubernetes\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.378822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378708 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-systemd\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.378822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378718 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b7e9986-2590-42d6-8fba-e49e780ebc72-cni-binary-copy\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.378822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378733 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-664l4\" (UniqueName: \"kubernetes.io/projected/859620d1-906d-4775-bf20-d888549445d4-kube-api-access-664l4\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.378822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378762 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwvk9\" (UniqueName: \"kubernetes.io/projected/b739e36f-5df5-4cec-8f18-498e13c693f7-kube-api-access-nwvk9\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.378822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378789 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-socket-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378836 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-system-cni-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378871 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-kubelet\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378895 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-systemd\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378922 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-ovn\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378928 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8636918-e4cf-4852-b6e7-640039dca7b8-env-overrides\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378947 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-os-release\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378934 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-daemon-config\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378978 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6lz\" (UniqueName: \"kubernetes.io/projected/51bf5173-3887-417e-a765-f9f7f0a2e099-kube-api-access-ql6lz\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379003 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-os-release\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379028 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-hostroot\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379050 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-sys\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379075 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-run-netns\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379098 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-var-lib-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379105 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-tuned\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379127 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-log-socket\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.378606 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-multus-conf-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.379179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379155 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379194 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-registration-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379221 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-systemd-units\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379247 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-cni-bin\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379277 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-multus-certs\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379303 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysconfig\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379329 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-node-log\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379380 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-kubernetes\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379391 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8636918-e4cf-4852-b6e7-640039dca7b8-ovn-node-metrics-cert\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379427 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-systemd\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379531 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-system-cni-dir\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379609 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-systemd\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379646 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-run-ovn\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379905 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-registration-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379936 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-node-log\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379980 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-kubelet\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.379988 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a375452-9057-4bd7-9337-422d42b662e0-agent-certs\") pod \"konnectivity-agent-cxvvr\" (UID: \"8a375452-9057-4bd7-9337-422d42b662e0\") " pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:12.380004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380001 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-systemd-units\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380042 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-cni-bin\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380042 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-host-run-netns\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380062 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-hostroot\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380070 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-var-lib-openvswitch\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380086 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-host-run-multus-certs\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380113 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-sys\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380214 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-tmp\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380227 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51bf5173-3887-417e-a765-f9f7f0a2e099-socket-dir\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380264 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8636918-e4cf-4852-b6e7-640039dca7b8-log-socket\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380278 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b7e9986-2590-42d6-8fba-e49e780ebc72-os-release\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.380787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.380305 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-etc-sysconfig\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.382348 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.382236 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8636918-e4cf-4852-b6e7-640039dca7b8-ovn-node-metrics-cert\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.384006 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.383977 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfmn5\" (UniqueName: \"kubernetes.io/projected/ec25af5a-4ef3-479c-8c81-0251ed117c8c-kube-api-access-mfmn5\") pod \"iptables-alerter-wrxx5\" (UID: \"ec25af5a-4ef3-479c-8c81-0251ed117c8c\") " pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.384578 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.384554 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4kkc\" (UniqueName: \"kubernetes.io/projected/b7bf0df9-24e9-4b81-9d28-9f59ed7ab952-kube-api-access-z4kkc\") pod \"tuned-fnhpv\" (UID: \"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952\") " pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.385361 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.385341 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j796\" (UniqueName: \"kubernetes.io/projected/f8636918-e4cf-4852-b6e7-640039dca7b8-kube-api-access-2j796\") pod \"ovnkube-node-4xppd\" (UID: \"f8636918-e4cf-4852-b6e7-640039dca7b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.386215 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.386195 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwxng\" (UniqueName: \"kubernetes.io/projected/98907b61-a541-4b22-8cd6-0f31f6baea96-kube-api-access-kwxng\") pod \"node-ca-26fj5\" (UID: \"98907b61-a541-4b22-8cd6-0f31f6baea96\") " pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.386629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.386602 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln44t\" (UniqueName: \"kubernetes.io/projected/8b7e9986-2590-42d6-8fba-e49e780ebc72-kube-api-access-ln44t\") pod \"multus-wxwcg\" (UID: \"8b7e9986-2590-42d6-8fba-e49e780ebc72\") " pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.387137 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.387086 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6lz\" (UniqueName: \"kubernetes.io/projected/51bf5173-3887-417e-a765-f9f7f0a2e099-kube-api-access-ql6lz\") pod \"aws-ebs-csi-driver-node-bgwhw\" (UID: \"51bf5173-3887-417e-a765-f9f7f0a2e099\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.387829 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.387808 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-664l4\" (UniqueName: \"kubernetes.io/projected/859620d1-906d-4775-bf20-d888549445d4-kube-api-access-664l4\") pod \"node-resolver-mc8fl\" (UID: \"859620d1-906d-4775-bf20-d888549445d4\") " pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.480337 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480252 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nwvk9\" (UniqueName: \"kubernetes.io/projected/b739e36f-5df5-4cec-8f18-498e13c693f7-kube-api-access-nwvk9\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.480337 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480301 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-os-release\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.480337 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480333 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480356 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wmkln\" (UniqueName: \"kubernetes.io/projected/386d5a57-cdc7-4d70-b3af-c06ea739c213-kube-api-access-wmkln\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480373 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-cni-binary-copy\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480392 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480423 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-system-cni-dir\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480453 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480480 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480543 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480570 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-cnibin\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.480629 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480623 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-system-cni-dir\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.481053 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480637 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-cnibin\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.481053 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.480750 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:12.481053 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.480840 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-os-release\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.481053 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.480882 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:12.980797459 +0000 UTC m=+3.070665694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:12.481053 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.481035 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.481053 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.481040 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b739e36f-5df5-4cec-8f18-498e13c693f7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.481305 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.481142 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-cni-binary-copy\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.481305 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.481213 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b739e36f-5df5-4cec-8f18-498e13c693f7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.487058 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.487022 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:21:12.487058 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.487058 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:21:12.487267 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.487082 2552 projected.go:194] Error preparing data for projected volume kube-api-access-bjtrw for pod openshift-network-diagnostics/network-check-target-kscb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:12.487267 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.487147 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw podName:d746ab9a-76e9-48bc-896f-f579b1923267 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:12.987128018 +0000 UTC m=+3.076996251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bjtrw" (UniqueName: "kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw") pod "network-check-target-kscb7" (UID: "d746ab9a-76e9-48bc-896f-f579b1923267") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:12.489384 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.489356 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmkln\" (UniqueName: \"kubernetes.io/projected/386d5a57-cdc7-4d70-b3af-c06ea739c213-kube-api-access-wmkln\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:12.489532 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.489441 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwvk9\" (UniqueName: \"kubernetes.io/projected/b739e36f-5df5-4cec-8f18-498e13c693f7-kube-api-access-nwvk9\") pod \"multus-additional-cni-plugins-pxw6v\" (UID: \"b739e36f-5df5-4cec-8f18-498e13c693f7\") " pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.565627 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.565591 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" Jun 02 19:21:12.573560 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.573536 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wxwcg" Jun 02 19:21:12.581195 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.581170 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wrxx5" Jun 02 19:21:12.586927 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.586903 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:12.592598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.592575 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:12.599230 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.599208 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" Jun 02 19:21:12.605845 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.605820 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mc8fl" Jun 02 19:21:12.612425 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.612406 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-26fj5" Jun 02 19:21:12.618044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.618021 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" Jun 02 19:21:12.834077 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.834050 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7bf0df9_24e9_4b81_9d28_9f59ed7ab952.slice/crio-78919a7db96f2e608a26ae4f8c9178e726e25e9aa4ac886bec2a3ee6d8b49d03 WatchSource:0}: Error finding container 78919a7db96f2e608a26ae4f8c9178e726e25e9aa4ac886bec2a3ee6d8b49d03: Status 404 returned error can't find the container with id 78919a7db96f2e608a26ae4f8c9178e726e25e9aa4ac886bec2a3ee6d8b49d03 Jun 02 19:21:12.836358 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.836330 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec25af5a_4ef3_479c_8c81_0251ed117c8c.slice/crio-79eb3241cfa5eba28bec4034dc34628a714350c281c1c5198a65f3f44d90b4bb WatchSource:0}: Error finding container 79eb3241cfa5eba28bec4034dc34628a714350c281c1c5198a65f3f44d90b4bb: Status 404 returned error can't find the container with id 79eb3241cfa5eba28bec4034dc34628a714350c281c1c5198a65f3f44d90b4bb Jun 02 19:21:12.839356 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.839329 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859620d1_906d_4775_bf20_d888549445d4.slice/crio-d943a795d27bc30a109a065ba03c9312a329b15172f9b4899e08f5e52a8280f8 WatchSource:0}: Error finding container d943a795d27bc30a109a065ba03c9312a329b15172f9b4899e08f5e52a8280f8: Status 404 returned error can't find the container with id d943a795d27bc30a109a065ba03c9312a329b15172f9b4899e08f5e52a8280f8 Jun 02 19:21:12.840573 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.840543 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7e9986_2590_42d6_8fba_e49e780ebc72.slice/crio-38c28a8c243ef0fc8f45fbfdff976bbd2bf21790b84c5272ab905af30308bcdf WatchSource:0}: Error finding container 38c28a8c243ef0fc8f45fbfdff976bbd2bf21790b84c5272ab905af30308bcdf: Status 404 returned error can't find the container with id 38c28a8c243ef0fc8f45fbfdff976bbd2bf21790b84c5272ab905af30308bcdf Jun 02 19:21:12.841102 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.841077 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb739e36f_5df5_4cec_8f18_498e13c693f7.slice/crio-6eee088373fea07b9783fccb9874898d4be43344ce68d16c7b8f8a37af8b595c WatchSource:0}: Error finding container 6eee088373fea07b9783fccb9874898d4be43344ce68d16c7b8f8a37af8b595c: Status 404 returned error can't find the container with id 6eee088373fea07b9783fccb9874898d4be43344ce68d16c7b8f8a37af8b595c Jun 02 19:21:12.843220 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.842428 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8636918_e4cf_4852_b6e7_640039dca7b8.slice/crio-4c54d0ccd56c959a27788b9bab28f0b1452fb3816bdc81bc9b10b68cde0271f8 WatchSource:0}: Error finding container 4c54d0ccd56c959a27788b9bab28f0b1452fb3816bdc81bc9b10b68cde0271f8: Status 404 returned error can't find the container with id 4c54d0ccd56c959a27788b9bab28f0b1452fb3816bdc81bc9b10b68cde0271f8 Jun 02 19:21:12.843435 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.843375 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a375452_9057_4bd7_9337_422d42b662e0.slice/crio-9ae5d2a7fc38f833fd327e81b33c71c9ea228d7d44c3ee5c91cb6fb68ec173d0 WatchSource:0}: Error finding container 9ae5d2a7fc38f833fd327e81b33c71c9ea228d7d44c3ee5c91cb6fb68ec173d0: Status 404 returned error can't find the container with id 9ae5d2a7fc38f833fd327e81b33c71c9ea228d7d44c3ee5c91cb6fb68ec173d0 Jun 02 19:21:12.845614 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.845586 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98907b61_a541_4b22_8cd6_0f31f6baea96.slice/crio-70a554f935375a1cf255a589e2b151f495ff7735c69765d103111c46e3caa400 WatchSource:0}: Error finding container 70a554f935375a1cf255a589e2b151f495ff7735c69765d103111c46e3caa400: Status 404 returned error can't find the container with id 70a554f935375a1cf255a589e2b151f495ff7735c69765d103111c46e3caa400 Jun 02 19:21:12.847456 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:12.847109 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51bf5173_3887_417e_a765_f9f7f0a2e099.slice/crio-93b4a8ef9f88612a436887e151a25ca148a103798905c9af1e534fa4e0073b34 WatchSource:0}: Error finding container 93b4a8ef9f88612a436887e151a25ca148a103798905c9af1e534fa4e0073b34: Status 404 returned error can't find the container with id 93b4a8ef9f88612a436887e151a25ca148a103798905c9af1e534fa4e0073b34 Jun 02 19:21:12.984225 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:12.983954 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:12.984399 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.984095 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:12.984399 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:12.984346 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:13.98432201 +0000 UTC m=+4.074190241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:13.085368 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.085332 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:13.085544 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:13.085482 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:21:13.085544 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:13.085529 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:21:13.085544 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:13.085543 2552 projected.go:194] Error preparing data for projected volume kube-api-access-bjtrw for pod openshift-network-diagnostics/network-check-target-kscb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:13.085715 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:13.085605 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw podName:d746ab9a-76e9-48bc-896f-f579b1923267 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:14.085586526 +0000 UTC m=+4.175454768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bjtrw" (UniqueName: "kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw") pod "network-check-target-kscb7" (UID: "d746ab9a-76e9-48bc-896f-f579b1923267") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:13.317648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.317601 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 19:16:11 +0000 UTC" deadline="2028-01-31 21:38:14.921642224 +0000 UTC" Jun 02 19:21:13.317648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.317646 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14594h17m1.60400076s" Jun 02 19:21:13.390534 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.389851 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:13.390534 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:13.389995 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:13.404211 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.404141 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wrxx5" event={"ID":"ec25af5a-4ef3-479c-8c81-0251ed117c8c","Type":"ContainerStarted","Data":"79eb3241cfa5eba28bec4034dc34628a714350c281c1c5198a65f3f44d90b4bb"} Jun 02 19:21:13.414767 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.414727 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" event={"ID":"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952","Type":"ContainerStarted","Data":"78919a7db96f2e608a26ae4f8c9178e726e25e9aa4ac886bec2a3ee6d8b49d03"} Jun 02 19:21:13.417232 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.417194 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" event={"ID":"51bf5173-3887-417e-a765-f9f7f0a2e099","Type":"ContainerStarted","Data":"93b4a8ef9f88612a436887e151a25ca148a103798905c9af1e534fa4e0073b34"} Jun 02 19:21:13.425518 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.425470 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-26fj5" event={"ID":"98907b61-a541-4b22-8cd6-0f31f6baea96","Type":"ContainerStarted","Data":"70a554f935375a1cf255a589e2b151f495ff7735c69765d103111c46e3caa400"} Jun 02 19:21:13.429671 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.429635 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerStarted","Data":"6eee088373fea07b9783fccb9874898d4be43344ce68d16c7b8f8a37af8b595c"} Jun 02 19:21:13.436580 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.436518 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mc8fl" event={"ID":"859620d1-906d-4775-bf20-d888549445d4","Type":"ContainerStarted","Data":"d943a795d27bc30a109a065ba03c9312a329b15172f9b4899e08f5e52a8280f8"} Jun 02 19:21:13.447985 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.447947 2552 generic.go:358] "Generic (PLEG): container finished" podID="2e335eccfebc403b8053a4893ae8bbe1" containerID="fe713ddd1b210bc8e74b6dc723edcdf3a78e46a08114492527ae954809d66102" exitCode=0 Jun 02 19:21:13.448136 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.448040 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" event={"ID":"2e335eccfebc403b8053a4893ae8bbe1","Type":"ContainerDied","Data":"fe713ddd1b210bc8e74b6dc723edcdf3a78e46a08114492527ae954809d66102"} Jun 02 19:21:13.468914 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.468551 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" event={"ID":"03340f265015a5ce3cd39b70624e15fb","Type":"ContainerStarted","Data":"e78dfe10cb2bca70e7d3cac028efb2bcf0368f0e413656ceb89523eb9c501cef"} Jun 02 19:21:13.474652 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.474515 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cxvvr" event={"ID":"8a375452-9057-4bd7-9337-422d42b662e0","Type":"ContainerStarted","Data":"9ae5d2a7fc38f833fd327e81b33c71c9ea228d7d44c3ee5c91cb6fb68ec173d0"} Jun 02 19:21:13.481533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.480546 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"4c54d0ccd56c959a27788b9bab28f0b1452fb3816bdc81bc9b10b68cde0271f8"} Jun 02 19:21:13.484020 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.483531 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wxwcg" event={"ID":"8b7e9986-2590-42d6-8fba-e49e780ebc72","Type":"ContainerStarted","Data":"38c28a8c243ef0fc8f45fbfdff976bbd2bf21790b84c5272ab905af30308bcdf"} Jun 02 19:21:13.489645 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.489588 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-231.ec2.internal" podStartSLOduration=2.489568671 podStartE2EDuration="2.489568671s" podCreationTimestamp="2026-06-02 19:21:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:21:13.489292646 +0000 UTC m=+3.579160883" watchObservedRunningTime="2026-06-02 19:21:13.489568671 +0000 UTC m=+3.579436905" Jun 02 19:21:13.994778 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:13.994563 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:13.994778 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:13.994729 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:13.995010 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:13.994797 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:15.994777196 +0000 UTC m=+6.084645412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:14.095802 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.095759 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:14.095990 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:14.095939 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:21:14.095990 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:14.095960 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:21:14.095990 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:14.095973 2552 projected.go:194] Error preparing data for projected volume kube-api-access-bjtrw for pod openshift-network-diagnostics/network-check-target-kscb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:14.096147 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:14.096038 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw podName:d746ab9a-76e9-48bc-896f-f579b1923267 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:16.096018886 +0000 UTC m=+6.185887118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bjtrw" (UniqueName: "kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw") pod "network-check-target-kscb7" (UID: "d746ab9a-76e9-48bc-896f-f579b1923267") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:14.390693 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.390659 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:14.391169 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:14.390791 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:14.424719 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.424685 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-tk8zj"] Jun 02 19:21:14.426468 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.426444 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:14.426623 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:14.426544 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:14.503482 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.503443 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:14.503689 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.503513 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/222e8269-50c6-471a-80fa-a9aedc1574ec-kubelet-config\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:14.503689 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.503542 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/222e8269-50c6-471a-80fa-a9aedc1574ec-dbus\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:14.520483 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.520441 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" event={"ID":"2e335eccfebc403b8053a4893ae8bbe1","Type":"ContainerStarted","Data":"5fe9377e48b61f8717296e61aa111370d2639a18523589ef06b2eda955cb59b9"} Jun 02 19:21:14.537416 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.537352 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-231.ec2.internal" podStartSLOduration=3.537331961 podStartE2EDuration="3.537331961s" podCreationTimestamp="2026-06-02 19:21:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:21:14.536756066 +0000 UTC m=+4.626624299" watchObservedRunningTime="2026-06-02 19:21:14.537331961 +0000 UTC m=+4.627200195" Jun 02 19:21:14.604268 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.604173 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:14.604268 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.604226 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/222e8269-50c6-471a-80fa-a9aedc1574ec-kubelet-config\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:14.604268 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.604247 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/222e8269-50c6-471a-80fa-a9aedc1574ec-dbus\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:14.604609 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.604566 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/222e8269-50c6-471a-80fa-a9aedc1574ec-dbus\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:14.604714 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:14.604695 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:14.604779 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:14.604767 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret podName:222e8269-50c6-471a-80fa-a9aedc1574ec nodeName:}" failed. No retries permitted until 2026-06-02 19:21:15.104746729 +0000 UTC m=+5.194614954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret") pod "global-pull-secret-syncer-tk8zj" (UID: "222e8269-50c6-471a-80fa-a9aedc1574ec") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:14.605070 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:14.605045 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/222e8269-50c6-471a-80fa-a9aedc1574ec-kubelet-config\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:15.110093 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:15.110051 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:15.110284 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:15.110214 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:15.110350 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:15.110286 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret podName:222e8269-50c6-471a-80fa-a9aedc1574ec nodeName:}" failed. No retries permitted until 2026-06-02 19:21:16.11026466 +0000 UTC m=+6.200132873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret") pod "global-pull-secret-syncer-tk8zj" (UID: "222e8269-50c6-471a-80fa-a9aedc1574ec") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:15.389368 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:15.389273 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:15.389598 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:15.389431 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:16.017750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:16.017709 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:16.018240 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.017934 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:16.018240 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.018004 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:20.017983023 +0000 UTC m=+10.107851239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:16.118557 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:16.118480 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:16.118751 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:16.118583 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:16.118751 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.118624 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:16.118751 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.118704 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:21:16.118751 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.118718 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret podName:222e8269-50c6-471a-80fa-a9aedc1574ec nodeName:}" failed. No retries permitted until 2026-06-02 19:21:18.118697679 +0000 UTC m=+8.208565895 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret") pod "global-pull-secret-syncer-tk8zj" (UID: "222e8269-50c6-471a-80fa-a9aedc1574ec") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:16.118751 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.118722 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:21:16.118751 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.118738 2552 projected.go:194] Error preparing data for projected volume kube-api-access-bjtrw for pod openshift-network-diagnostics/network-check-target-kscb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:16.119033 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.118788 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw podName:d746ab9a-76e9-48bc-896f-f579b1923267 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:20.11877248 +0000 UTC m=+10.208640695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bjtrw" (UniqueName: "kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw") pod "network-check-target-kscb7" (UID: "d746ab9a-76e9-48bc-896f-f579b1923267") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:16.389552 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:16.389404 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:16.389709 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.389561 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:16.389709 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:16.389672 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:16.389827 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:16.389787 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:17.390148 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:17.389603 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:17.390148 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:17.389754 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:18.136801 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:18.136714 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:18.137001 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:18.136859 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:18.137001 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:18.136948 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret podName:222e8269-50c6-471a-80fa-a9aedc1574ec nodeName:}" failed. No retries permitted until 2026-06-02 19:21:22.136924282 +0000 UTC m=+12.226792523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret") pod "global-pull-secret-syncer-tk8zj" (UID: "222e8269-50c6-471a-80fa-a9aedc1574ec") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:18.389892 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:18.389742 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:18.389892 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:18.389766 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:18.390111 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:18.389900 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:18.390111 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:18.389965 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:19.389871 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:19.389830 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:19.390333 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:19.389988 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:20.052479 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:20.052372 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:20.052687 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:20.052560 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:20.052687 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:20.052636 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:28.052616721 +0000 UTC m=+18.142484936 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:20.153797 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:20.153128 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:20.153797 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:20.153311 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:21:20.153797 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:20.153329 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:21:20.153797 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:20.153342 2552 projected.go:194] Error preparing data for projected volume kube-api-access-bjtrw for pod openshift-network-diagnostics/network-check-target-kscb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:20.153797 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:20.153403 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw podName:d746ab9a-76e9-48bc-896f-f579b1923267 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:28.153383455 +0000 UTC m=+18.243251680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bjtrw" (UniqueName: "kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw") pod "network-check-target-kscb7" (UID: "d746ab9a-76e9-48bc-896f-f579b1923267") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:20.390572 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:20.390469 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:20.391100 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:20.390612 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:20.391100 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:20.390973 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:20.391100 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:20.391079 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:21.389644 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:21.389606 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:21.389843 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:21.389756 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:22.170100 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:22.170063 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:22.170534 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:22.170282 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:22.170534 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:22.170366 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret podName:222e8269-50c6-471a-80fa-a9aedc1574ec nodeName:}" failed. No retries permitted until 2026-06-02 19:21:30.170345499 +0000 UTC m=+20.260213726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret") pod "global-pull-secret-syncer-tk8zj" (UID: "222e8269-50c6-471a-80fa-a9aedc1574ec") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:22.390086 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:22.390047 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:22.390086 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:22.390081 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:22.390327 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:22.390189 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:22.390383 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:22.390317 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:23.389234 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:23.389150 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:23.389703 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:23.389284 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:24.389213 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:24.389181 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:24.389405 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:24.389313 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:24.389405 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:24.389352 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:24.389826 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:24.389469 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:25.389970 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:25.389918 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:25.390458 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:25.390070 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:26.390038 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:26.390000 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:26.390516 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:26.390048 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:26.390516 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:26.390130 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:26.390516 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:26.390240 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:27.389674 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:27.389633 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:27.389888 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:27.389758 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:28.116100 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:28.116061 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:28.116615 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:28.116200 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:28.116615 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:28.116259 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:44.11624199 +0000 UTC m=+34.206110218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:28.216735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:28.216685 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:28.216899 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:28.216880 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:21:28.216957 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:28.216904 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:21:28.216957 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:28.216923 2552 projected.go:194] Error preparing data for projected volume kube-api-access-bjtrw for pod openshift-network-diagnostics/network-check-target-kscb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:28.217025 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:28.216984 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw podName:d746ab9a-76e9-48bc-896f-f579b1923267 nodeName:}" failed. No retries permitted until 2026-06-02 19:21:44.216969061 +0000 UTC m=+34.306837271 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bjtrw" (UniqueName: "kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw") pod "network-check-target-kscb7" (UID: "d746ab9a-76e9-48bc-896f-f579b1923267") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:28.392137 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:28.392045 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:28.392292 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:28.392158 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:28.392292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:28.392218 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:28.392391 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:28.392309 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:29.389344 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:29.389306 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:29.389967 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:29.389426 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:30.233217 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:30.233179 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:30.233400 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:30.233312 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:30.233400 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:30.233375 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret podName:222e8269-50c6-471a-80fa-a9aedc1574ec nodeName:}" failed. No retries permitted until 2026-06-02 19:21:46.233358221 +0000 UTC m=+36.323226432 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret") pod "global-pull-secret-syncer-tk8zj" (UID: "222e8269-50c6-471a-80fa-a9aedc1574ec") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:21:30.391929 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:30.391900 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:30.391929 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:30.391915 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:30.392435 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:30.391998 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:30.392435 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:30.392060 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:31.389770 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.389725 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:31.389919 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:31.389894 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:31.552374 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.552082 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" event={"ID":"51bf5173-3887-417e-a765-f9f7f0a2e099","Type":"ContainerStarted","Data":"35f32f20f04f5134107377d8754cbe9e337b4c0c87e1f052cc7a8c5b3035d9b4"} Jun 02 19:21:31.554134 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.554093 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-26fj5" event={"ID":"98907b61-a541-4b22-8cd6-0f31f6baea96","Type":"ContainerStarted","Data":"ee9c7f1c343700e3be97c02cc97bf72487d066599822693a2778084d2d8952f3"} Jun 02 19:21:31.556432 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.556384 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerStarted","Data":"f60339f612a217836fe8d95fb1ef1bbc47382d8c208b381591a39bd9973a0aae"} Jun 02 19:21:31.558279 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.558244 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mc8fl" event={"ID":"859620d1-906d-4775-bf20-d888549445d4","Type":"ContainerStarted","Data":"31798ad8bb091514c75e42610272a07930e4b19454d603e3c290b31aca8c9b31"} Jun 02 19:21:31.560096 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.560068 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cxvvr" event={"ID":"8a375452-9057-4bd7-9337-422d42b662e0","Type":"ContainerStarted","Data":"7fcf95b665ba16e8e71ae319777a240d8b35dd3d71b9fec3f5bce954985efc42"} Jun 02 19:21:31.562025 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.561997 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"15980c94ad3d44c6f357eb0783557be845169d46bb6a421ed71f2325331ecbff"} Jun 02 19:21:31.563842 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.563817 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wxwcg" event={"ID":"8b7e9986-2590-42d6-8fba-e49e780ebc72","Type":"ContainerStarted","Data":"c1df0eda91370c3314cc76648a08e54fb5afde773f660685b7eaaba1edd379ce"} Jun 02 19:21:31.565414 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.565389 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" event={"ID":"b7bf0df9-24e9-4b81-9d28-9f59ed7ab952","Type":"ContainerStarted","Data":"94818ad8f84f62fc7276caf041459e7f22c4ec1d4657540203a8c1eb069241f2"} Jun 02 19:21:31.571058 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.570998 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-26fj5" podStartSLOduration=3.121431311 podStartE2EDuration="21.570980733s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.849000528 +0000 UTC m=+2.938868738" lastFinishedPulling="2026-06-02 19:21:31.298549939 +0000 UTC m=+21.388418160" observedRunningTime="2026-06-02 19:21:31.570482623 +0000 UTC m=+21.660350854" watchObservedRunningTime="2026-06-02 19:21:31.570980733 +0000 UTC m=+21.660848968" Jun 02 19:21:31.587783 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.587734 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-wxwcg" podStartSLOduration=3.078328706 podStartE2EDuration="21.58771857s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.842693372 +0000 UTC m=+2.932561583" lastFinishedPulling="2026-06-02 19:21:31.352083221 +0000 UTC m=+21.441951447" observedRunningTime="2026-06-02 19:21:31.587280878 +0000 UTC m=+21.677149114" watchObservedRunningTime="2026-06-02 19:21:31.58771857 +0000 UTC m=+21.677586803" Jun 02 19:21:31.601596 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.601507 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-mc8fl" podStartSLOduration=3.144068871 podStartE2EDuration="21.601474321s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.841143386 +0000 UTC m=+2.931011598" lastFinishedPulling="2026-06-02 19:21:31.298548831 +0000 UTC m=+21.388417048" observedRunningTime="2026-06-02 19:21:31.601356113 +0000 UTC m=+21.691224346" watchObservedRunningTime="2026-06-02 19:21:31.601474321 +0000 UTC m=+21.691342556" Jun 02 19:21:31.615394 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.615139 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cxvvr" podStartSLOduration=7.891450413 podStartE2EDuration="21.615111025s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.847034914 +0000 UTC m=+2.936903131" lastFinishedPulling="2026-06-02 19:21:26.570695517 +0000 UTC m=+16.660563743" observedRunningTime="2026-06-02 19:21:31.615088003 +0000 UTC m=+21.704956243" watchObservedRunningTime="2026-06-02 19:21:31.615111025 +0000 UTC m=+21.704979240" Jun 02 19:21:31.635551 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:31.635477 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fnhpv" podStartSLOduration=3.17139044 podStartE2EDuration="21.635465086s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.836355233 +0000 UTC m=+2.926223448" lastFinishedPulling="2026-06-02 19:21:31.300429866 +0000 UTC m=+21.390298094" observedRunningTime="2026-06-02 19:21:31.635180112 +0000 UTC m=+21.725048343" watchObservedRunningTime="2026-06-02 19:21:31.635465086 +0000 UTC m=+21.725333319" Jun 02 19:21:32.389430 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.389395 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:32.389637 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.389391 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:32.389637 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:32.389544 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:32.389637 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:32.389580 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:32.569931 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.569891 2552 generic.go:358] "Generic (PLEG): container finished" podID="b739e36f-5df5-4cec-8f18-498e13c693f7" containerID="f60339f612a217836fe8d95fb1ef1bbc47382d8c208b381591a39bd9973a0aae" exitCode=0 Jun 02 19:21:32.570447 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.569981 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerDied","Data":"f60339f612a217836fe8d95fb1ef1bbc47382d8c208b381591a39bd9973a0aae"} Jun 02 19:21:32.572602 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.572583 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:21:32.572897 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.572874 2552 generic.go:358] "Generic (PLEG): container finished" podID="f8636918-e4cf-4852-b6e7-640039dca7b8" containerID="ea58c337476bb8aed00599ebad86e2ffdc989ffe6031fece339736d0cf6fbb3b" exitCode=1 Jun 02 19:21:32.573017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.572991 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"ac70f5701c34f13e1e5679fd5ffcdfdd7530664eab7752c0c6f90ab0ba7394d7"} Jun 02 19:21:32.573098 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.573029 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"55acd2665d25130ff610383ba91fb38c450eb222e5de4a1a9c6b20dae0b84b49"} Jun 02 19:21:32.573098 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.573044 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"d66da9623d5c34dc07fbb39a18be6e3d972b4d345875c52a4660343a0c537d6a"} Jun 02 19:21:32.573098 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.573054 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"de791fa9a0b6fc04992176cdf69ab1a9a493ba0dbc76c0323e5e4287edfff745"} Jun 02 19:21:32.573098 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.573061 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerDied","Data":"ea58c337476bb8aed00599ebad86e2ffdc989ffe6031fece339736d0cf6fbb3b"} Jun 02 19:21:32.734557 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:32.734531 2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 19:21:33.335504 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:33.335383 2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T19:21:32.734551433Z","UUID":"7bbd4647-2f43-4691-b172-e9d75ff92df3","Handler":null,"Name":"","Endpoint":""} Jun 02 19:21:33.338006 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:33.337978 2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 19:21:33.338144 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:33.338015 2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 19:21:33.389850 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:33.389811 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:33.390039 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:33.389968 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:33.576621 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:33.576552 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" event={"ID":"51bf5173-3887-417e-a765-f9f7f0a2e099","Type":"ContainerStarted","Data":"2d5487bb8cceb3351149a6dcda185807fef285a58e7dac8ffc55eb74de886203"} Jun 02 19:21:33.578252 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:33.578219 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wrxx5" event={"ID":"ec25af5a-4ef3-479c-8c81-0251ed117c8c","Type":"ContainerStarted","Data":"69a7a5b5092d38bf6bfa1c0d9b8fd39c68fbe3f31970aacead6d514b1411d882"} Jun 02 19:21:33.592754 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:33.592658 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wrxx5" podStartSLOduration=5.132618352 podStartE2EDuration="23.59264233s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.838427805 +0000 UTC m=+2.928296018" lastFinishedPulling="2026-06-02 19:21:31.298451781 +0000 UTC m=+21.388319996" observedRunningTime="2026-06-02 19:21:33.592418793 +0000 UTC m=+23.682287028" watchObservedRunningTime="2026-06-02 19:21:33.59264233 +0000 UTC m=+23.682510565" Jun 02 19:21:34.390342 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:34.390077 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:34.390519 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:34.390123 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:34.390519 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:34.390394 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:34.390519 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:34.390439 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:34.459647 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:34.459613 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:34.460261 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:34.460228 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:34.583578 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:34.583547 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:21:34.584008 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:34.583956 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"ce9c099994422e3457b90f94414732ecc5d7e75ddc57d9f9c97ce36dadc1dc9a"} Jun 02 19:21:34.585928 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:34.585899 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" event={"ID":"51bf5173-3887-417e-a765-f9f7f0a2e099","Type":"ContainerStarted","Data":"cc57aafdb9b8975f9dc33e2e2f6921b96e9058957fa81a5178503958abbe934b"} Jun 02 19:21:34.604763 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:34.604717 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgwhw" podStartSLOduration=3.671374289 podStartE2EDuration="24.604701998s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.849779585 +0000 UTC m=+2.939647808" lastFinishedPulling="2026-06-02 19:21:33.783107306 +0000 UTC m=+23.872975517" observedRunningTime="2026-06-02 19:21:34.604303033 +0000 UTC m=+24.694171266" watchObservedRunningTime="2026-06-02 19:21:34.604701998 +0000 UTC m=+24.694570314" Jun 02 19:21:35.389469 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:35.389431 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:35.389695 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:35.389583 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:36.389519 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.389422 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:36.389519 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.389457 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:36.391472 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:36.389585 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:36.391472 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:36.389706 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:36.592789 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.592755 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:21:36.593186 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.593159 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"fb2ca6850f598227cd5ef0be8205a054a2ac8e63ec63e3a8f862dedefd618aac"} Jun 02 19:21:36.594458 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.594430 2552 scope.go:117] "RemoveContainer" containerID="ea58c337476bb8aed00599ebad86e2ffdc989ffe6031fece339736d0cf6fbb3b" Jun 02 19:21:36.594568 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.594535 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:36.594768 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.594752 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:36.594840 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.594794 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:36.615830 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.615557 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:36.616221 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:36.616189 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:21:37.389149 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:37.389106 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:37.389360 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:37.389254 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:37.630856 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:37.630754 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:21:37.632758 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:37.631566 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" event={"ID":"f8636918-e4cf-4852-b6e7-640039dca7b8","Type":"ContainerStarted","Data":"7bacd7f6c6a8f47cfab9bcb4852ea874b4d50dbdb86aa39246cc3fe3149aea9b"} Jun 02 19:21:37.660219 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:37.660120 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" podStartSLOduration=9.136479275 podStartE2EDuration="27.660104778s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.845006455 +0000 UTC m=+2.934874666" lastFinishedPulling="2026-06-02 19:21:31.368631942 +0000 UTC m=+21.458500169" observedRunningTime="2026-06-02 19:21:37.659622312 +0000 UTC m=+27.749490545" watchObservedRunningTime="2026-06-02 19:21:37.660104778 +0000 UTC m=+27.749973089" Jun 02 19:21:38.290617 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:38.290587 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sgj2b"] Jun 02 19:21:38.290819 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:38.290700 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:38.290819 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:38.290804 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:38.294619 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:38.294572 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kscb7"] Jun 02 19:21:38.294758 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:38.294733 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:38.294866 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:38.294840 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:38.295351 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:38.295324 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tk8zj"] Jun 02 19:21:38.295520 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:38.295438 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:38.295597 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:38.295557 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:39.636622 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:39.636587 2552 generic.go:358] "Generic (PLEG): container finished" podID="b739e36f-5df5-4cec-8f18-498e13c693f7" containerID="67189a0f6eb3d5410df95a15d7019d9f73d9183f16489f24486f5cd238ed466e" exitCode=0 Jun 02 19:21:39.636988 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:39.636638 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerDied","Data":"67189a0f6eb3d5410df95a15d7019d9f73d9183f16489f24486f5cd238ed466e"} Jun 02 19:21:40.390262 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:40.390219 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:40.390441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:40.390306 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:40.390441 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:40.390316 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:40.390562 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:40.390441 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:40.390562 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:40.390481 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:40.390636 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:40.390578 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:40.532308 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:40.532265 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:40.532474 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:40.532453 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 02 19:21:40.533155 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:40.533109 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cxvvr" Jun 02 19:21:41.646030 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:41.645849 2552 generic.go:358] "Generic (PLEG): container finished" podID="b739e36f-5df5-4cec-8f18-498e13c693f7" containerID="2eeb153ca5ae07dae513ef27914bb9e01d9bf5af08cb656113a90099f4ebe5a6" exitCode=0 Jun 02 19:21:41.646440 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:41.645928 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerDied","Data":"2eeb153ca5ae07dae513ef27914bb9e01d9bf5af08cb656113a90099f4ebe5a6"} Jun 02 19:21:42.389352 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:42.389277 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:42.389482 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:42.389378 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tk8zj" podUID="222e8269-50c6-471a-80fa-a9aedc1574ec" Jun 02 19:21:42.389482 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:42.389398 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:42.389482 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:42.389424 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:42.389607 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:42.389538 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kscb7" podUID="d746ab9a-76e9-48bc-896f-f579b1923267" Jun 02 19:21:42.389666 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:42.389646 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:21:43.652257 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:43.652221 2552 generic.go:358] "Generic (PLEG): container finished" podID="b739e36f-5df5-4cec-8f18-498e13c693f7" containerID="2f36cbf06a3833f35a11a96f1b0810087db29e2de45ae18cf64e93fae0273ce7" exitCode=0 Jun 02 19:21:43.652846 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:43.652284 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerDied","Data":"2f36cbf06a3833f35a11a96f1b0810087db29e2de45ae18cf64e93fae0273ce7"} Jun 02 19:21:44.140231 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.140196 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:44.140417 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.140336 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:44.140417 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.140394 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:22:16.14037924 +0000 UTC m=+66.230247450 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:21:44.205889 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.205849 2552 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-231.ec2.internal" event="NodeReady" Jun 02 19:21:44.206067 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.206018 2552 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 19:21:44.241164 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.241127 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:44.241371 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.241300 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:21:44.241371 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.241328 2552 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:21:44.241371 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.241341 2552 projected.go:194] Error preparing data for projected volume kube-api-access-bjtrw for pod openshift-network-diagnostics/network-check-target-kscb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:44.241544 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.241410 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw podName:d746ab9a-76e9-48bc-896f-f579b1923267 nodeName:}" failed. No retries permitted until 2026-06-02 19:22:16.241389964 +0000 UTC m=+66.331258200 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bjtrw" (UniqueName: "kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw") pod "network-check-target-kscb7" (UID: "d746ab9a-76e9-48bc-896f-f579b1923267") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:21:44.251704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.251667 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5k45w"] Jun 02 19:21:44.274380 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.274342 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zb24h"] Jun 02 19:21:44.274567 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.274543 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.277113 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.277083 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 19:21:44.277251 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.277162 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 19:21:44.277368 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.277352 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sz4sv\"" Jun 02 19:21:44.298451 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.298414 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5k45w"] Jun 02 19:21:44.298451 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.298451 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zb24h"] Jun 02 19:21:44.298703 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.298594 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:44.301204 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.301174 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 19:21:44.301371 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.301285 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bv8mp\"" Jun 02 19:21:44.301371 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.301312 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 19:21:44.301621 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.301602 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 19:21:44.389998 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.389962 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:21:44.390194 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.389963 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:44.390194 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.390174 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:21:44.392590 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.392459 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9qt55\"" Jun 02 19:21:44.392735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.392632 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tb5vh\"" Jun 02 19:21:44.392735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.392726 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 19:21:44.392735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.392733 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 19:21:44.392909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.392774 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 19:21:44.392909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.392864 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 19:21:44.442742 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.442704 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:44.442742 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.442753 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab4d72b-4346-4824-8058-f14f5de2fbab-tmp-dir\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.443021 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.442805 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4ab4d72b-4346-4824-8058-f14f5de2fbab-config-volume\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.443021 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.442841 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j44z5\" (UniqueName: \"kubernetes.io/projected/6bb96f33-bcb4-4a7e-b879-159149ced68e-kube-api-access-j44z5\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:44.443021 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.442913 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xct5\" (UniqueName: \"kubernetes.io/projected/4ab4d72b-4346-4824-8058-f14f5de2fbab-kube-api-access-7xct5\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.443021 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.442966 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.544201 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.544159 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7xct5\" (UniqueName: \"kubernetes.io/projected/4ab4d72b-4346-4824-8058-f14f5de2fbab-kube-api-access-7xct5\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.544201 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.544201 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.544412 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.544305 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:21:44.544412 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.544333 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:44.544412 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.544385 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls podName:4ab4d72b-4346-4824-8058-f14f5de2fbab nodeName:}" failed. No retries permitted until 2026-06-02 19:21:45.044363829 +0000 UTC m=+35.134232054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls") pod "dns-default-5k45w" (UID: "4ab4d72b-4346-4824-8058-f14f5de2fbab") : secret "dns-default-metrics-tls" not found Jun 02 19:21:44.544602 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.544422 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab4d72b-4346-4824-8058-f14f5de2fbab-tmp-dir\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.544602 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.544470 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4ab4d72b-4346-4824-8058-f14f5de2fbab-config-volume\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.544602 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.544484 2552 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:21:44.544602 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.544545 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j44z5\" (UniqueName: \"kubernetes.io/projected/6bb96f33-bcb4-4a7e-b879-159149ced68e-kube-api-access-j44z5\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:44.544602 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:44.544574 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert podName:6bb96f33-bcb4-4a7e-b879-159149ced68e nodeName:}" failed. No retries permitted until 2026-06-02 19:21:45.044555883 +0000 UTC m=+35.134424098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert") pod "ingress-canary-zb24h" (UID: "6bb96f33-bcb4-4a7e-b879-159149ced68e") : secret "canary-serving-cert" not found Jun 02 19:21:44.544894 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.544877 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab4d72b-4346-4824-8058-f14f5de2fbab-tmp-dir\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.545128 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.545109 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4ab4d72b-4346-4824-8058-f14f5de2fbab-config-volume\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.556581 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.556549 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xct5\" (UniqueName: \"kubernetes.io/projected/4ab4d72b-4346-4824-8058-f14f5de2fbab-kube-api-access-7xct5\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:44.556919 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:44.556895 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j44z5\" (UniqueName: \"kubernetes.io/projected/6bb96f33-bcb4-4a7e-b879-159149ced68e-kube-api-access-j44z5\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:45.048579 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:45.048537 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:45.049216 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:45.048713 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:21:45.049216 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:45.048728 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:45.049216 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:45.048793 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls podName:4ab4d72b-4346-4824-8058-f14f5de2fbab nodeName:}" failed. No retries permitted until 2026-06-02 19:21:46.048769522 +0000 UTC m=+36.138637738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls") pod "dns-default-5k45w" (UID: "4ab4d72b-4346-4824-8058-f14f5de2fbab") : secret "dns-default-metrics-tls" not found Jun 02 19:21:45.049216 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:45.048853 2552 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:21:45.049216 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:45.048927 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert podName:6bb96f33-bcb4-4a7e-b879-159149ced68e nodeName:}" failed. No retries permitted until 2026-06-02 19:21:46.048909903 +0000 UTC m=+36.138778131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert") pod "ingress-canary-zb24h" (UID: "6bb96f33-bcb4-4a7e-b879-159149ced68e") : secret "canary-serving-cert" not found Jun 02 19:21:46.057270 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:46.057227 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:46.057769 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:46.057286 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:46.057769 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:46.057411 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:21:46.057769 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:46.057422 2552 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:21:46.057769 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:46.057525 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls podName:4ab4d72b-4346-4824-8058-f14f5de2fbab nodeName:}" failed. No retries permitted until 2026-06-02 19:21:48.057484345 +0000 UTC m=+38.147352563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls") pod "dns-default-5k45w" (UID: "4ab4d72b-4346-4824-8058-f14f5de2fbab") : secret "dns-default-metrics-tls" not found Jun 02 19:21:46.057769 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:46.057547 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert podName:6bb96f33-bcb4-4a7e-b879-159149ced68e nodeName:}" failed. No retries permitted until 2026-06-02 19:21:48.057533761 +0000 UTC m=+38.147401972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert") pod "ingress-canary-zb24h" (UID: "6bb96f33-bcb4-4a7e-b879-159149ced68e") : secret "canary-serving-cert" not found Jun 02 19:21:46.259805 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:46.259769 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:46.262374 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:46.262348 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/222e8269-50c6-471a-80fa-a9aedc1574ec-original-pull-secret\") pod \"global-pull-secret-syncer-tk8zj\" (UID: \"222e8269-50c6-471a-80fa-a9aedc1574ec\") " pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:46.515077 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:46.515041 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tk8zj" Jun 02 19:21:46.699424 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:46.699375 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tk8zj"] Jun 02 19:21:46.705772 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:21:46.705738 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222e8269_50c6_471a_80fa_a9aedc1574ec.slice/crio-bed1296c0d2b922877814594f4108fa88a30107b6969b585f38eb4fc23bfae20 WatchSource:0}: Error finding container bed1296c0d2b922877814594f4108fa88a30107b6969b585f38eb4fc23bfae20: Status 404 returned error can't find the container with id bed1296c0d2b922877814594f4108fa88a30107b6969b585f38eb4fc23bfae20 Jun 02 19:21:47.662331 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:47.662209 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tk8zj" event={"ID":"222e8269-50c6-471a-80fa-a9aedc1574ec","Type":"ContainerStarted","Data":"bed1296c0d2b922877814594f4108fa88a30107b6969b585f38eb4fc23bfae20"} Jun 02 19:21:48.076727 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:48.076681 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:48.076927 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:48.076743 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:48.076927 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:48.076886 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:21:48.077045 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:48.076976 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls podName:4ab4d72b-4346-4824-8058-f14f5de2fbab nodeName:}" failed. No retries permitted until 2026-06-02 19:21:52.076952125 +0000 UTC m=+42.166820343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls") pod "dns-default-5k45w" (UID: "4ab4d72b-4346-4824-8058-f14f5de2fbab") : secret "dns-default-metrics-tls" not found Jun 02 19:21:48.077045 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:48.076896 2552 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:21:48.077146 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:48.077045 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert podName:6bb96f33-bcb4-4a7e-b879-159149ced68e nodeName:}" failed. No retries permitted until 2026-06-02 19:21:52.077027376 +0000 UTC m=+42.166895588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert") pod "ingress-canary-zb24h" (UID: "6bb96f33-bcb4-4a7e-b879-159149ced68e") : secret "canary-serving-cert" not found Jun 02 19:21:50.674837 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:50.674736 2552 generic.go:358] "Generic (PLEG): container finished" podID="b739e36f-5df5-4cec-8f18-498e13c693f7" containerID="c17aa084abf6137a871c99f973aeb127914a1e3491be66382c1d358f3e742c19" exitCode=0 Jun 02 19:21:50.674837 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:50.674814 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerDied","Data":"c17aa084abf6137a871c99f973aeb127914a1e3491be66382c1d358f3e742c19"} Jun 02 19:21:52.106585 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:52.106382 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:21:52.106992 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:52.106610 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:21:52.106992 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:52.106540 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:21:52.106992 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:52.106732 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls podName:4ab4d72b-4346-4824-8058-f14f5de2fbab nodeName:}" failed. No retries permitted until 2026-06-02 19:22:00.106708246 +0000 UTC m=+50.196576477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls") pod "dns-default-5k45w" (UID: "4ab4d72b-4346-4824-8058-f14f5de2fbab") : secret "dns-default-metrics-tls" not found Jun 02 19:21:52.106992 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:52.106743 2552 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:21:52.106992 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:21:52.106791 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert podName:6bb96f33-bcb4-4a7e-b879-159149ced68e nodeName:}" failed. No retries permitted until 2026-06-02 19:22:00.106779188 +0000 UTC m=+50.196647399 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert") pod "ingress-canary-zb24h" (UID: "6bb96f33-bcb4-4a7e-b879-159149ced68e") : secret "canary-serving-cert" not found Jun 02 19:21:52.683158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:52.683068 2552 generic.go:358] "Generic (PLEG): container finished" podID="b739e36f-5df5-4cec-8f18-498e13c693f7" containerID="da79a70cc03b76f51ca66a7f895790cdb456dd6f7cd297a2bb2d08776cbbff02" exitCode=0 Jun 02 19:21:52.683317 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:52.683153 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerDied","Data":"da79a70cc03b76f51ca66a7f895790cdb456dd6f7cd297a2bb2d08776cbbff02"} Jun 02 19:21:52.684383 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:52.684351 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tk8zj" event={"ID":"222e8269-50c6-471a-80fa-a9aedc1574ec","Type":"ContainerStarted","Data":"7ca9e2ed3ddef62d412d47b32856da77df7b1849e25af270035f7f8ecb791d90"} Jun 02 19:21:52.726839 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:52.726782 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tk8zj" podStartSLOduration=33.073946205 podStartE2EDuration="38.726762683s" podCreationTimestamp="2026-06-02 19:21:14 +0000 UTC" firstStartedPulling="2026-06-02 19:21:46.707966716 +0000 UTC m=+36.797834930" lastFinishedPulling="2026-06-02 19:21:52.360783194 +0000 UTC m=+42.450651408" observedRunningTime="2026-06-02 19:21:52.726218141 +0000 UTC m=+42.816086374" watchObservedRunningTime="2026-06-02 19:21:52.726762683 +0000 UTC m=+42.816630919" Jun 02 19:21:53.689601 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:53.689551 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" event={"ID":"b739e36f-5df5-4cec-8f18-498e13c693f7","Type":"ContainerStarted","Data":"2e0bcc712ef8b468ae87e8cfe4e8c96139a0fde7a5263815b6388d41edba4135"} Jun 02 19:21:53.720259 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:21:53.720209 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pxw6v" podStartSLOduration=6.432024086 podStartE2EDuration="43.72019447s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:21:12.843108342 +0000 UTC m=+2.932976563" lastFinishedPulling="2026-06-02 19:21:50.131278733 +0000 UTC m=+40.221146947" observedRunningTime="2026-06-02 19:21:53.718765413 +0000 UTC m=+43.808633645" watchObservedRunningTime="2026-06-02 19:21:53.72019447 +0000 UTC m=+43.810062703" Jun 02 19:22:00.163646 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:00.163607 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:22:00.163646 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:00.163651 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:22:00.164281 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:00.163752 2552 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:22:00.164281 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:00.163757 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:22:00.164281 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:00.163808 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert podName:6bb96f33-bcb4-4a7e-b879-159149ced68e nodeName:}" failed. No retries permitted until 2026-06-02 19:22:16.163792317 +0000 UTC m=+66.253660528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert") pod "ingress-canary-zb24h" (UID: "6bb96f33-bcb4-4a7e-b879-159149ced68e") : secret "canary-serving-cert" not found Jun 02 19:22:00.164281 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:00.163820 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls podName:4ab4d72b-4346-4824-8058-f14f5de2fbab nodeName:}" failed. No retries permitted until 2026-06-02 19:22:16.163814027 +0000 UTC m=+66.253682238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls") pod "dns-default-5k45w" (UID: "4ab4d72b-4346-4824-8058-f14f5de2fbab") : secret "dns-default-metrics-tls" not found Jun 02 19:22:08.644624 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:08.644585 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xppd" Jun 02 19:22:16.179679 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.179627 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:22:16.179679 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.179693 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:22:16.180105 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.179717 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:22:16.180105 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:16.179836 2552 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:22:16.180105 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:16.179902 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert podName:6bb96f33-bcb4-4a7e-b879-159149ced68e nodeName:}" failed. No retries permitted until 2026-06-02 19:22:48.179883965 +0000 UTC m=+98.269752187 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert") pod "ingress-canary-zb24h" (UID: "6bb96f33-bcb4-4a7e-b879-159149ced68e") : secret "canary-serving-cert" not found Jun 02 19:22:16.180105 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:16.179837 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:22:16.180105 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:16.179995 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls podName:4ab4d72b-4346-4824-8058-f14f5de2fbab nodeName:}" failed. No retries permitted until 2026-06-02 19:22:48.179980756 +0000 UTC m=+98.269848972 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls") pod "dns-default-5k45w" (UID: "4ab4d72b-4346-4824-8058-f14f5de2fbab") : secret "dns-default-metrics-tls" not found Jun 02 19:22:16.182200 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.182183 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 19:22:16.190177 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:16.190158 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 19:22:16.190245 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:16.190234 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:20.190212838 +0000 UTC m=+130.280081061 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : secret "metrics-daemon-secret" not found Jun 02 19:22:16.280290 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.280241 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:22:16.283227 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.283211 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 19:22:16.293703 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.293686 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 19:22:16.305536 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.305508 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjtrw\" (UniqueName: \"kubernetes.io/projected/d746ab9a-76e9-48bc-896f-f579b1923267-kube-api-access-bjtrw\") pod \"network-check-target-kscb7\" (UID: \"d746ab9a-76e9-48bc-896f-f579b1923267\") " pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:22:16.512653 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.512565 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9qt55\"" Jun 02 19:22:16.519736 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.519715 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:22:16.642711 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.642679 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kscb7"] Jun 02 19:22:16.646048 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:22:16.646016 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd746ab9a_76e9_48bc_896f_f579b1923267.slice/crio-a2378af67337a431d778c1912d36ecbf6cfa2bf53e715db6c4a9a70bd5b8b1fb WatchSource:0}: Error finding container a2378af67337a431d778c1912d36ecbf6cfa2bf53e715db6c4a9a70bd5b8b1fb: Status 404 returned error can't find the container with id a2378af67337a431d778c1912d36ecbf6cfa2bf53e715db6c4a9a70bd5b8b1fb Jun 02 19:22:16.736677 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:16.736646 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kscb7" event={"ID":"d746ab9a-76e9-48bc-896f-f579b1923267","Type":"ContainerStarted","Data":"a2378af67337a431d778c1912d36ecbf6cfa2bf53e715db6c4a9a70bd5b8b1fb"} Jun 02 19:22:19.743915 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:19.743814 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kscb7" event={"ID":"d746ab9a-76e9-48bc-896f-f579b1923267","Type":"ContainerStarted","Data":"76298ed3011502cd6a09c7f29bfc2107136d1ff3060e65485f9dfb79550e0b5d"} Jun 02 19:22:19.744304 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:19.744013 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:22:19.759860 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:19.759806 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kscb7" podStartSLOduration=66.996774829 podStartE2EDuration="1m9.759791809s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:22:16.647890733 +0000 UTC m=+66.737758945" lastFinishedPulling="2026-06-02 19:22:19.410907714 +0000 UTC m=+69.500775925" observedRunningTime="2026-06-02 19:22:19.758985072 +0000 UTC m=+69.848853305" watchObservedRunningTime="2026-06-02 19:22:19.759791809 +0000 UTC m=+69.849660079" Jun 02 19:22:48.198869 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:48.198766 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:22:48.198869 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:48.198811 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:22:48.199309 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:48.198903 2552 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 19:22:48.199309 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:48.198918 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 19:22:48.199309 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:48.198956 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert podName:6bb96f33-bcb4-4a7e-b879-159149ced68e nodeName:}" failed. No retries permitted until 2026-06-02 19:23:52.198942118 +0000 UTC m=+162.288810329 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert") pod "ingress-canary-zb24h" (UID: "6bb96f33-bcb4-4a7e-b879-159149ced68e") : secret "canary-serving-cert" not found Jun 02 19:22:48.199309 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:22:48.198982 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls podName:4ab4d72b-4346-4824-8058-f14f5de2fbab nodeName:}" failed. No retries permitted until 2026-06-02 19:23:52.198968417 +0000 UTC m=+162.288836633 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls") pod "dns-default-5k45w" (UID: "4ab4d72b-4346-4824-8058-f14f5de2fbab") : secret "dns-default-metrics-tls" not found Jun 02 19:22:50.747955 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:22:50.747917 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kscb7" Jun 02 19:23:20.221031 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:20.220961 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:23:20.221587 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:20.221124 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 19:23:20.221587 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:20.221204 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs podName:386d5a57-cdc7-4d70-b3af-c06ea739c213 nodeName:}" failed. No retries permitted until 2026-06-02 19:25:22.221184601 +0000 UTC m=+252.311052812 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs") pod "network-metrics-daemon-sgj2b" (UID: "386d5a57-cdc7-4d70-b3af-c06ea739c213") : secret "metrics-daemon-secret" not found Jun 02 19:23:30.383719 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.383685 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w"] Jun 02 19:23:30.386633 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.386611 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-f5qhp"] Jun 02 19:23:30.386763 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.386749 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.388995 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.388978 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 02 19:23:30.389106 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.389073 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:23:30.389106 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.389091 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 02 19:23:30.389242 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.389222 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 02 19:23:30.389352 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.389336 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fkjx7\"" Jun 02 19:23:30.389452 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.389439 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.391516 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.391487 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 02 19:23:30.391647 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.391629 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:23:30.391905 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.391888 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 02 19:23:30.392066 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.392047 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-62h49\"" Jun 02 19:23:30.392356 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.392338 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 02 19:23:30.395511 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.395473 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w"] Jun 02 19:23:30.397661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.397642 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-f5qhp"] Jun 02 19:23:30.398610 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.398592 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 02 19:23:30.478545 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.478514 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97"] Jun 02 19:23:30.481332 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.481314 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97" Jun 02 19:23:30.488286 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.488265 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wbrwv\"" Jun 02 19:23:30.488398 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.488289 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 02 19:23:30.488777 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.488762 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:23:30.493713 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.493697 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/238f4e41-37e5-454b-84fb-12af646cc790-serving-cert\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.493783 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.493724 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkfxx\" (UniqueName: \"kubernetes.io/projected/238f4e41-37e5-454b-84fb-12af646cc790-kube-api-access-nkfxx\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.493783 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.493753 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/238f4e41-37e5-454b-84fb-12af646cc790-trusted-ca\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.493859 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.493810 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e563216c-9d47-4016-b694-2fc740bccd5d-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.493859 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.493848 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e563216c-9d47-4016-b694-2fc740bccd5d-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.493935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.493910 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7545\" (UniqueName: \"kubernetes.io/projected/e563216c-9d47-4016-b694-2fc740bccd5d-kube-api-access-h7545\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.493976 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.493935 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/238f4e41-37e5-454b-84fb-12af646cc790-config\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.499405 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.499387 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-59d8544f99-c22p5"] Jun 02 19:23:30.502047 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.502034 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.517474 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:30.517422 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-dockercfg-f96tb\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f96tb\"" type="*v1.Secret" Jun 02 19:23:30.517577 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:30.517536 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-ingress-cert\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" Jun 02 19:23:30.517577 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:30.517535 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-stats-default\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" Jun 02 19:23:30.519123 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:30.519105 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca-bundle\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" type="*v1.ConfigMap" Jun 02 19:23:30.519123 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:30.519116 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jun 02 19:23:30.519250 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:30.519207 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jun 02 19:23:30.519250 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:30.519204 2552 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-metrics-certs-default\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" Jun 02 19:23:30.536272 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.536249 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97"] Jun 02 19:23:30.571124 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.571103 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59d8544f99-c22p5"] Jun 02 19:23:30.594170 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594144 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/238f4e41-37e5-454b-84fb-12af646cc790-config\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.594272 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594181 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.594272 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594211 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/238f4e41-37e5-454b-84fb-12af646cc790-serving-cert\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.594272 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594237 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nkfxx\" (UniqueName: \"kubernetes.io/projected/238f4e41-37e5-454b-84fb-12af646cc790-kube-api-access-nkfxx\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.594431 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594275 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/238f4e41-37e5-454b-84fb-12af646cc790-trusted-ca\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.594431 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594323 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e563216c-9d47-4016-b694-2fc740bccd5d-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.594431 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594351 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-default-certificate\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.594431 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594379 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-stats-auth\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.594431 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594405 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srtw2\" (UniqueName: \"kubernetes.io/projected/3a06f660-d2cd-419b-9b5c-32109faa93b4-kube-api-access-srtw2\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.594688 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594482 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e563216c-9d47-4016-b694-2fc740bccd5d-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.594688 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594542 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.594688 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594569 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f974t\" (UniqueName: \"kubernetes.io/projected/076f8656-2a48-47cf-8b9e-017008bab7ff-kube-api-access-f974t\") pod \"volume-data-source-validator-7598fd5bd-w5k97\" (UID: \"076f8656-2a48-47cf-8b9e-017008bab7ff\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97" Jun 02 19:23:30.594688 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594632 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h7545\" (UniqueName: \"kubernetes.io/projected/e563216c-9d47-4016-b694-2fc740bccd5d-kube-api-access-h7545\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.594919 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.594892 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/238f4e41-37e5-454b-84fb-12af646cc790-config\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.595122 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.595100 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e563216c-9d47-4016-b694-2fc740bccd5d-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.595246 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.595225 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/238f4e41-37e5-454b-84fb-12af646cc790-trusted-ca\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.596601 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.596579 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e563216c-9d47-4016-b694-2fc740bccd5d-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.596776 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.596756 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/238f4e41-37e5-454b-84fb-12af646cc790-serving-cert\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.608971 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.608945 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkfxx\" (UniqueName: \"kubernetes.io/projected/238f4e41-37e5-454b-84fb-12af646cc790-kube-api-access-nkfxx\") pod \"console-operator-86d7c6cc77-f5qhp\" (UID: \"238f4e41-37e5-454b-84fb-12af646cc790\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.608971 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.608959 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7545\" (UniqueName: \"kubernetes.io/projected/e563216c-9d47-4016-b694-2fc740bccd5d-kube-api-access-h7545\") pod \"kube-storage-version-migrator-operator-96ddc787c-fk87w\" (UID: \"e563216c-9d47-4016-b694-2fc740bccd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.695387 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.695296 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-stats-auth\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.695387 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.695353 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-srtw2\" (UniqueName: \"kubernetes.io/projected/3a06f660-d2cd-419b-9b5c-32109faa93b4-kube-api-access-srtw2\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.695584 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.695398 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.695584 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.695423 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f974t\" (UniqueName: \"kubernetes.io/projected/076f8656-2a48-47cf-8b9e-017008bab7ff-kube-api-access-f974t\") pod \"volume-data-source-validator-7598fd5bd-w5k97\" (UID: \"076f8656-2a48-47cf-8b9e-017008bab7ff\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97" Jun 02 19:23:30.695584 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.695479 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.695584 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.695543 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-default-certificate\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:30.698282 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.698266 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" Jun 02 19:23:30.703331 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.703313 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:30.704172 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.704154 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f974t\" (UniqueName: \"kubernetes.io/projected/076f8656-2a48-47cf-8b9e-017008bab7ff-kube-api-access-f974t\") pod \"volume-data-source-validator-7598fd5bd-w5k97\" (UID: \"076f8656-2a48-47cf-8b9e-017008bab7ff\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97" Jun 02 19:23:30.789340 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.789307 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97" Jun 02 19:23:30.824741 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.824713 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w"] Jun 02 19:23:30.827104 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:23:30.827073 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode563216c_9d47_4016_b694_2fc740bccd5d.slice/crio-7b9dfdb1ea5957afbdb1e532ca56a54b8606a7a46c5a8eb9bfcb551870d157be WatchSource:0}: Error finding container 7b9dfdb1ea5957afbdb1e532ca56a54b8606a7a46c5a8eb9bfcb551870d157be: Status 404 returned error can't find the container with id 7b9dfdb1ea5957afbdb1e532ca56a54b8606a7a46c5a8eb9bfcb551870d157be Jun 02 19:23:30.846789 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.846640 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-f5qhp"] Jun 02 19:23:30.848858 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:23:30.848828 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238f4e41_37e5_454b_84fb_12af646cc790.slice/crio-6b504487752e4a7c5af79bb79d621760cb86baed9fc4dfe07b9209e5b1574bd2 WatchSource:0}: Error finding container 6b504487752e4a7c5af79bb79d621760cb86baed9fc4dfe07b9209e5b1574bd2: Status 404 returned error can't find the container with id 6b504487752e4a7c5af79bb79d621760cb86baed9fc4dfe07b9209e5b1574bd2 Jun 02 19:23:30.887711 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.887679 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" event={"ID":"238f4e41-37e5-454b-84fb-12af646cc790","Type":"ContainerStarted","Data":"6b504487752e4a7c5af79bb79d621760cb86baed9fc4dfe07b9209e5b1574bd2"} Jun 02 19:23:30.888762 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.888726 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" event={"ID":"e563216c-9d47-4016-b694-2fc740bccd5d","Type":"ContainerStarted","Data":"7b9dfdb1ea5957afbdb1e532ca56a54b8606a7a46c5a8eb9bfcb551870d157be"} Jun 02 19:23:30.922434 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:30.922404 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97"] Jun 02 19:23:30.925697 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:23:30.925677 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076f8656_2a48_47cf_8b9e_017008bab7ff.slice/crio-b397d3632c2f5b4bac32536a0bbfc9b8514dd6fec5bfd3dd305d8d55e22c0ff7 WatchSource:0}: Error finding container b397d3632c2f5b4bac32536a0bbfc9b8514dd6fec5bfd3dd305d8d55e22c0ff7: Status 404 returned error can't find the container with id b397d3632c2f5b4bac32536a0bbfc9b8514dd6fec5bfd3dd305d8d55e22c0ff7 Jun 02 19:23:31.378688 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.378655 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f96tb\"" Jun 02 19:23:31.494765 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.494733 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 02 19:23:31.495886 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:31.495860 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:31.995835212 +0000 UTC m=+142.085703447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : configmap references non-existent config key: service-ca.crt Jun 02 19:23:31.513775 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.513737 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 19:23:31.520444 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.520415 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 02 19:23:31.529704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.529603 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-default-certificate\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:31.588416 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.588385 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 02 19:23:31.595816 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:31.595782 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 19:23:31.595955 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:31.595865 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:32.095843797 +0000 UTC m=+142.185712009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : secret "router-metrics-certs-default" not found Jun 02 19:23:31.634534 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.634438 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 19:23:31.647218 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.647164 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srtw2\" (UniqueName: \"kubernetes.io/projected/3a06f660-d2cd-419b-9b5c-32109faa93b4-kube-api-access-srtw2\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:31.696339 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:31.696312 2552 secret.go:189] Couldn't get secret openshift-ingress/router-stats-default: failed to sync secret cache: timed out waiting for the condition Jun 02 19:23:31.696485 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:31.696407 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-stats-auth podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:32.196382003 +0000 UTC m=+142.286250218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "stats-auth" (UniqueName: "kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-stats-auth") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : failed to sync secret cache: timed out waiting for the condition Jun 02 19:23:31.840396 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.840359 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 02 19:23:31.893094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:31.893010 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97" event={"ID":"076f8656-2a48-47cf-8b9e-017008bab7ff","Type":"ContainerStarted","Data":"b397d3632c2f5b4bac32536a0bbfc9b8514dd6fec5bfd3dd305d8d55e22c0ff7"} Jun 02 19:23:32.006917 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:32.006878 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:32.007100 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:32.007044 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:33.007023906 +0000 UTC m=+143.096892117 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : configmap references non-existent config key: service-ca.crt Jun 02 19:23:32.108166 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:32.108132 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:32.108349 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:32.108291 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 19:23:32.108428 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:32.108358 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:33.108340263 +0000 UTC m=+143.198208473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : secret "router-metrics-certs-default" not found Jun 02 19:23:32.209675 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:32.209573 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-stats-auth\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:32.212526 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:32.212481 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-stats-auth\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:33.015782 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.015741 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:33.016251 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:33.015897 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:35.01587568 +0000 UTC m=+145.105743891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : configmap references non-existent config key: service-ca.crt Jun 02 19:23:33.117104 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.117068 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:33.117305 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:33.117189 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 19:23:33.117305 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:33.117248 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:35.117233573 +0000 UTC m=+145.207101784 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : secret "router-metrics-certs-default" not found Jun 02 19:23:33.899200 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.899173 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/0.log" Jun 02 19:23:33.899377 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.899213 2552 generic.go:358] "Generic (PLEG): container finished" podID="238f4e41-37e5-454b-84fb-12af646cc790" containerID="6b57d46828cccc11adab0b27309553a491637761ad5ed238db5d3e80a0390752" exitCode=255 Jun 02 19:23:33.899377 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.899267 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" event={"ID":"238f4e41-37e5-454b-84fb-12af646cc790","Type":"ContainerDied","Data":"6b57d46828cccc11adab0b27309553a491637761ad5ed238db5d3e80a0390752"} Jun 02 19:23:33.899601 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.899582 2552 scope.go:117] "RemoveContainer" containerID="6b57d46828cccc11adab0b27309553a491637761ad5ed238db5d3e80a0390752" Jun 02 19:23:33.900640 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.900602 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" event={"ID":"e563216c-9d47-4016-b694-2fc740bccd5d","Type":"ContainerStarted","Data":"f3ab26f0fbaf88ed62cf361d992b4571f38f5b754a2b91345dd25adf41bde1f2"} Jun 02 19:23:33.901842 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.901809 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97" event={"ID":"076f8656-2a48-47cf-8b9e-017008bab7ff","Type":"ContainerStarted","Data":"39353e14d775ca253a781130cb3bbda9475129ab6f4344ffcec524928e3ee9a4"} Jun 02 19:23:33.937265 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:33.937211 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" podStartSLOduration=1.386794606 podStartE2EDuration="3.937191892s" podCreationTimestamp="2026-06-02 19:23:30 +0000 UTC" firstStartedPulling="2026-06-02 19:23:30.829353412 +0000 UTC m=+140.919221623" lastFinishedPulling="2026-06-02 19:23:33.379750681 +0000 UTC m=+143.469618909" observedRunningTime="2026-06-02 19:23:33.936749594 +0000 UTC m=+144.026617827" watchObservedRunningTime="2026-06-02 19:23:33.937191892 +0000 UTC m=+144.027060127" Jun 02 19:23:34.906767 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:34.906734 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/1.log" Jun 02 19:23:34.907161 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:34.907090 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/0.log" Jun 02 19:23:34.907161 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:34.907125 2552 generic.go:358] "Generic (PLEG): container finished" podID="238f4e41-37e5-454b-84fb-12af646cc790" containerID="1ba4beba848df38468a3aba7d1754217c31e070c0c88092d2742543de0b6cb9b" exitCode=255 Jun 02 19:23:34.907244 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:34.907212 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" event={"ID":"238f4e41-37e5-454b-84fb-12af646cc790","Type":"ContainerDied","Data":"1ba4beba848df38468a3aba7d1754217c31e070c0c88092d2742543de0b6cb9b"} Jun 02 19:23:34.907289 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:34.907259 2552 scope.go:117] "RemoveContainer" containerID="6b57d46828cccc11adab0b27309553a491637761ad5ed238db5d3e80a0390752" Jun 02 19:23:34.907681 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:34.907662 2552 scope.go:117] "RemoveContainer" containerID="1ba4beba848df38468a3aba7d1754217c31e070c0c88092d2742543de0b6cb9b" Jun 02 19:23:34.907844 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:34.907824 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f5qhp_openshift-console-operator(238f4e41-37e5-454b-84fb-12af646cc790)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" podUID="238f4e41-37e5-454b-84fb-12af646cc790" Jun 02 19:23:34.936390 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:34.936321 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-w5k97" podStartSLOduration=2.483230029 podStartE2EDuration="4.936302922s" podCreationTimestamp="2026-06-02 19:23:30 +0000 UTC" firstStartedPulling="2026-06-02 19:23:30.927389117 +0000 UTC m=+141.017257328" lastFinishedPulling="2026-06-02 19:23:33.380462003 +0000 UTC m=+143.470330221" observedRunningTime="2026-06-02 19:23:33.961317388 +0000 UTC m=+144.051185621" watchObservedRunningTime="2026-06-02 19:23:34.936302922 +0000 UTC m=+145.026171156" Jun 02 19:23:35.033740 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:35.033695 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:35.033943 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:35.033852 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:39.033827403 +0000 UTC m=+149.123695628 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : configmap references non-existent config key: service-ca.crt Jun 02 19:23:35.134286 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:35.134245 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:35.134416 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:35.134382 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 19:23:35.134459 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:35.134447 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:39.134428542 +0000 UTC m=+149.224296756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : secret "router-metrics-certs-default" not found Jun 02 19:23:35.910949 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:35.910921 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/1.log" Jun 02 19:23:35.911400 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:35.911196 2552 scope.go:117] "RemoveContainer" containerID="1ba4beba848df38468a3aba7d1754217c31e070c0c88092d2742543de0b6cb9b" Jun 02 19:23:35.911400 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:35.911358 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f5qhp_openshift-console-operator(238f4e41-37e5-454b-84fb-12af646cc790)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" podUID="238f4e41-37e5-454b-84fb-12af646cc790" Jun 02 19:23:36.555463 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.555436 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mc8fl_859620d1-906d-4775-bf20-d888549445d4/dns-node-resolver/0.log" Jun 02 19:23:36.592853 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.592827 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-n9lnt"] Jun 02 19:23:36.596987 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.596959 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.599749 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.599732 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 02 19:23:36.600818 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.600723 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 02 19:23:36.600818 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.600766 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 02 19:23:36.600818 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.600723 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-crfc2\"" Jun 02 19:23:36.601023 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.600770 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 02 19:23:36.603205 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.603183 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-n9lnt"] Jun 02 19:23:36.748086 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.748049 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3bc60f9a-c842-47c3-9126-1dcad86e8e28-signing-key\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.748086 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.748088 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdwh7\" (UniqueName: \"kubernetes.io/projected/3bc60f9a-c842-47c3-9126-1dcad86e8e28-kube-api-access-jdwh7\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.748321 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.748182 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3bc60f9a-c842-47c3-9126-1dcad86e8e28-signing-cabundle\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.849098 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.849069 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3bc60f9a-c842-47c3-9126-1dcad86e8e28-signing-key\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.849277 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.849106 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jdwh7\" (UniqueName: \"kubernetes.io/projected/3bc60f9a-c842-47c3-9126-1dcad86e8e28-kube-api-access-jdwh7\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.849349 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.849279 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3bc60f9a-c842-47c3-9126-1dcad86e8e28-signing-cabundle\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.849901 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.849880 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3bc60f9a-c842-47c3-9126-1dcad86e8e28-signing-cabundle\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.851412 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.851388 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3bc60f9a-c842-47c3-9126-1dcad86e8e28-signing-key\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.859187 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.859162 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdwh7\" (UniqueName: \"kubernetes.io/projected/3bc60f9a-c842-47c3-9126-1dcad86e8e28-kube-api-access-jdwh7\") pod \"service-ca-685df94d57-n9lnt\" (UID: \"3bc60f9a-c842-47c3-9126-1dcad86e8e28\") " pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:36.907125 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:36.907090 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-n9lnt" Jun 02 19:23:37.034618 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:37.034586 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-n9lnt"] Jun 02 19:23:37.037240 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:23:37.037202 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc60f9a_c842_47c3_9126_1dcad86e8e28.slice/crio-bbdddbc0b6aa9e425f23381de8302d6ae1fe458ffd95317daa05bf2a320bddb6 WatchSource:0}: Error finding container bbdddbc0b6aa9e425f23381de8302d6ae1fe458ffd95317daa05bf2a320bddb6: Status 404 returned error can't find the container with id bbdddbc0b6aa9e425f23381de8302d6ae1fe458ffd95317daa05bf2a320bddb6 Jun 02 19:23:37.158374 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:37.158294 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-26fj5_98907b61-a541-4b22-8cd6-0f31f6baea96/node-ca/0.log" Jun 02 19:23:37.918345 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:37.918307 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-n9lnt" event={"ID":"3bc60f9a-c842-47c3-9126-1dcad86e8e28","Type":"ContainerStarted","Data":"bbdddbc0b6aa9e425f23381de8302d6ae1fe458ffd95317daa05bf2a320bddb6"} Jun 02 19:23:38.922504 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:38.922451 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-n9lnt" event={"ID":"3bc60f9a-c842-47c3-9126-1dcad86e8e28","Type":"ContainerStarted","Data":"5ff96e72398eadbd2cbfdf416ece2c222ec8f1011ad9ba319c443fd18d4557ad"} Jun 02 19:23:38.939355 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:38.939303 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-n9lnt" podStartSLOduration=1.1863160719999999 podStartE2EDuration="2.939286245s" podCreationTimestamp="2026-06-02 19:23:36 +0000 UTC" firstStartedPulling="2026-06-02 19:23:37.038905642 +0000 UTC m=+147.128773867" lastFinishedPulling="2026-06-02 19:23:38.791875826 +0000 UTC m=+148.881744040" observedRunningTime="2026-06-02 19:23:38.938508856 +0000 UTC m=+149.028377082" watchObservedRunningTime="2026-06-02 19:23:38.939286245 +0000 UTC m=+149.029154477" Jun 02 19:23:39.068765 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:39.068710 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:39.068934 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:39.068860 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:47.068841459 +0000 UTC m=+157.158709681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : configmap references non-existent config key: service-ca.crt Jun 02 19:23:39.169090 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:39.169058 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:39.169229 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:39.169166 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 19:23:39.169229 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:39.169217 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs podName:3a06f660-d2cd-419b-9b5c-32109faa93b4 nodeName:}" failed. No retries permitted until 2026-06-02 19:23:47.169203246 +0000 UTC m=+157.259071456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs") pod "router-default-59d8544f99-c22p5" (UID: "3a06f660-d2cd-419b-9b5c-32109faa93b4") : secret "router-metrics-certs-default" not found Jun 02 19:23:40.704060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:40.704009 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:40.704060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:40.704068 2552 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:23:40.704533 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:40.704409 2552 scope.go:117] "RemoveContainer" containerID="1ba4beba848df38468a3aba7d1754217c31e070c0c88092d2742543de0b6cb9b" Jun 02 19:23:40.704616 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:40.704598 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f5qhp_openshift-console-operator(238f4e41-37e5-454b-84fb-12af646cc790)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" podUID="238f4e41-37e5-454b-84fb-12af646cc790" Jun 02 19:23:47.135822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.135763 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:47.136422 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.136399 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a06f660-d2cd-419b-9b5c-32109faa93b4-service-ca-bundle\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:47.236935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.236892 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:47.239292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.239263 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a06f660-d2cd-419b-9b5c-32109faa93b4-metrics-certs\") pod \"router-default-59d8544f99-c22p5\" (UID: \"3a06f660-d2cd-419b-9b5c-32109faa93b4\") " pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:47.286041 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:47.285990 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5k45w" podUID="4ab4d72b-4346-4824-8058-f14f5de2fbab" Jun 02 19:23:47.310132 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.310106 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:47.310223 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:47.310171 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-zb24h" podUID="6bb96f33-bcb4-4a7e-b879-159149ced68e" Jun 02 19:23:47.401985 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:47.401939 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-sgj2b" podUID="386d5a57-cdc7-4d70-b3af-c06ea739c213" Jun 02 19:23:47.444457 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.444416 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59d8544f99-c22p5"] Jun 02 19:23:47.448180 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:23:47.448145 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a06f660_d2cd_419b_9b5c_32109faa93b4.slice/crio-ad51e1593c559e5a68b95301baeca913b51b51a6866055eafd528bbf4fbaba86 WatchSource:0}: Error finding container ad51e1593c559e5a68b95301baeca913b51b51a6866055eafd528bbf4fbaba86: Status 404 returned error can't find the container with id ad51e1593c559e5a68b95301baeca913b51b51a6866055eafd528bbf4fbaba86 Jun 02 19:23:47.946176 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.946131 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59d8544f99-c22p5" event={"ID":"3a06f660-d2cd-419b-9b5c-32109faa93b4","Type":"ContainerStarted","Data":"8b5c40fe710089b527dcb259007fbef18398312b52f43251b1fa1d98b46fac02"} Jun 02 19:23:47.946176 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.946179 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59d8544f99-c22p5" event={"ID":"3a06f660-d2cd-419b-9b5c-32109faa93b4","Type":"ContainerStarted","Data":"ad51e1593c559e5a68b95301baeca913b51b51a6866055eafd528bbf4fbaba86"} Jun 02 19:23:47.946428 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.946201 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5k45w" Jun 02 19:23:47.964978 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:47.964920 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-59d8544f99-c22p5" podStartSLOduration=17.964905346 podStartE2EDuration="17.964905346s" podCreationTimestamp="2026-06-02 19:23:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:23:47.964037054 +0000 UTC m=+158.053905289" watchObservedRunningTime="2026-06-02 19:23:47.964905346 +0000 UTC m=+158.054773578" Jun 02 19:23:48.310792 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:48.310680 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:48.313356 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:48.313330 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:48.949022 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:48.948993 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:48.950333 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:48.950309 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-59d8544f99-c22p5" Jun 02 19:23:52.278795 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:52.278747 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:23:52.278795 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:52.278796 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:23:52.281141 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:52.281111 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ab4d72b-4346-4824-8058-f14f5de2fbab-metrics-tls\") pod \"dns-default-5k45w\" (UID: \"4ab4d72b-4346-4824-8058-f14f5de2fbab\") " pod="openshift-dns/dns-default-5k45w" Jun 02 19:23:52.281259 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:52.281210 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bb96f33-bcb4-4a7e-b879-159149ced68e-cert\") pod \"ingress-canary-zb24h\" (UID: \"6bb96f33-bcb4-4a7e-b879-159149ced68e\") " pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:23:52.449505 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:52.449458 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sz4sv\"" Jun 02 19:23:52.457561 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:52.457543 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5k45w" Jun 02 19:23:52.582169 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:52.582135 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5k45w"] Jun 02 19:23:52.585901 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:23:52.585855 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab4d72b_4346_4824_8058_f14f5de2fbab.slice/crio-6c86f0dff4f68e066dda9cde12f69a62159eece5b2bbc55502648f2f5a89e5a2 WatchSource:0}: Error finding container 6c86f0dff4f68e066dda9cde12f69a62159eece5b2bbc55502648f2f5a89e5a2: Status 404 returned error can't find the container with id 6c86f0dff4f68e066dda9cde12f69a62159eece5b2bbc55502648f2f5a89e5a2 Jun 02 19:23:52.959177 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:52.959135 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5k45w" event={"ID":"4ab4d72b-4346-4824-8058-f14f5de2fbab","Type":"ContainerStarted","Data":"6c86f0dff4f68e066dda9cde12f69a62159eece5b2bbc55502648f2f5a89e5a2"} Jun 02 19:23:54.964550 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:54.964513 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5k45w" event={"ID":"4ab4d72b-4346-4824-8058-f14f5de2fbab","Type":"ContainerStarted","Data":"e803a05e5529a3ec23f0a364f1e3f143ddffe9d400d529b4ade8a7b1bc6a8ffd"} Jun 02 19:23:54.965012 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:54.964557 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5k45w" event={"ID":"4ab4d72b-4346-4824-8058-f14f5de2fbab","Type":"ContainerStarted","Data":"7bcca3dfa78ea0a14f5700f5207e9afdb3da37a6e93e09cfc3db5e64e0ab4b01"} Jun 02 19:23:54.965012 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:54.964695 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5k45w" Jun 02 19:23:54.980745 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:54.980678 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-5k45w" podStartSLOduration=129.616658208 podStartE2EDuration="2m10.980661746s" podCreationTimestamp="2026-06-02 19:21:44 +0000 UTC" firstStartedPulling="2026-06-02 19:23:52.58774042 +0000 UTC m=+162.677608634" lastFinishedPulling="2026-06-02 19:23:53.951743961 +0000 UTC m=+164.041612172" observedRunningTime="2026-06-02 19:23:54.980535681 +0000 UTC m=+165.070403914" watchObservedRunningTime="2026-06-02 19:23:54.980661746 +0000 UTC m=+165.070529978" Jun 02 19:23:55.406842 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.406806 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-98cf6659c-sfh8t"] Jun 02 19:23:55.409803 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.409784 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.412735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.412707 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 19:23:55.413029 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.412779 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 19:23:55.413029 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.412944 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-42tmx\"" Jun 02 19:23:55.413209 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.413199 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 19:23:55.420245 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.420219 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 19:23:55.429414 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.429390 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-98cf6659c-sfh8t"] Jun 02 19:23:55.430260 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.430239 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r747v"] Jun 02 19:23:55.433388 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.433369 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.436053 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.436034 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 19:23:55.436268 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.436034 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 19:23:55.436350 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.436087 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 19:23:55.436350 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.436095 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 19:23:55.436462 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.436131 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lq9p9\"" Jun 02 19:23:55.444177 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.444154 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r747v"] Jun 02 19:23:55.503893 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.503862 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93d0c8df-f627-41d4-918c-0e60643554ec-installation-pull-secrets\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.504052 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.503902 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93d0c8df-f627-41d4-918c-0e60643554ec-image-registry-private-configuration\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.504052 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.503927 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93d0c8df-f627-41d4-918c-0e60643554ec-registry-certificates\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.504052 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.503954 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-bound-sa-token\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.504052 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.504017 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-registry-tls\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.504409 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.504132 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcgfd\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-kube-api-access-dcgfd\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.504409 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.504166 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93d0c8df-f627-41d4-918c-0e60643554ec-ca-trust-extracted\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.504409 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.504192 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93d0c8df-f627-41d4-918c-0e60643554ec-trusted-ca\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604557 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604527 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-registry-tls\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604570 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6dcacfc-140d-4bed-bb93-18a8436546fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.604737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604596 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6dcacfc-140d-4bed-bb93-18a8436546fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.604737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604621 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6dcacfc-140d-4bed-bb93-18a8436546fb-data-volume\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.604737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604680 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e6dcacfc-140d-4bed-bb93-18a8436546fb-crio-socket\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.604737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604740 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dcgfd\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-kube-api-access-dcgfd\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604763 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93d0c8df-f627-41d4-918c-0e60643554ec-ca-trust-extracted\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604784 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93d0c8df-f627-41d4-918c-0e60643554ec-trusted-ca\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604810 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93d0c8df-f627-41d4-918c-0e60643554ec-installation-pull-secrets\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604839 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93d0c8df-f627-41d4-918c-0e60643554ec-image-registry-private-configuration\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604882 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93d0c8df-f627-41d4-918c-0e60643554ec-registry-certificates\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604906 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-bound-sa-token\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.604990 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.604934 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wssl7\" (UniqueName: \"kubernetes.io/projected/e6dcacfc-140d-4bed-bb93-18a8436546fb-kube-api-access-wssl7\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.605325 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.605196 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93d0c8df-f627-41d4-918c-0e60643554ec-ca-trust-extracted\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.605882 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.605854 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93d0c8df-f627-41d4-918c-0e60643554ec-trusted-ca\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.606721 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.606698 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93d0c8df-f627-41d4-918c-0e60643554ec-registry-certificates\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.607144 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.607118 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-registry-tls\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.607442 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.607420 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93d0c8df-f627-41d4-918c-0e60643554ec-installation-pull-secrets\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.607483 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.607443 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93d0c8df-f627-41d4-918c-0e60643554ec-image-registry-private-configuration\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.614094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.614070 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-bound-sa-token\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.614189 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.614080 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcgfd\" (UniqueName: \"kubernetes.io/projected/93d0c8df-f627-41d4-918c-0e60643554ec-kube-api-access-dcgfd\") pod \"image-registry-98cf6659c-sfh8t\" (UID: \"93d0c8df-f627-41d4-918c-0e60643554ec\") " pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.705367 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.705299 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6dcacfc-140d-4bed-bb93-18a8436546fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.705367 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.705330 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6dcacfc-140d-4bed-bb93-18a8436546fb-data-volume\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.705367 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.705353 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e6dcacfc-140d-4bed-bb93-18a8436546fb-crio-socket\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.705582 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.705449 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wssl7\" (UniqueName: \"kubernetes.io/projected/e6dcacfc-140d-4bed-bb93-18a8436546fb-kube-api-access-wssl7\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.705582 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.705483 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6dcacfc-140d-4bed-bb93-18a8436546fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.705582 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.705528 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e6dcacfc-140d-4bed-bb93-18a8436546fb-crio-socket\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.705684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.705634 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6dcacfc-140d-4bed-bb93-18a8436546fb-data-volume\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.705945 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.705925 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6dcacfc-140d-4bed-bb93-18a8436546fb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.707701 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.707680 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6dcacfc-140d-4bed-bb93-18a8436546fb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.713450 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.713428 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wssl7\" (UniqueName: \"kubernetes.io/projected/e6dcacfc-140d-4bed-bb93-18a8436546fb-kube-api-access-wssl7\") pod \"insights-runtime-extractor-r747v\" (UID: \"e6dcacfc-140d-4bed-bb93-18a8436546fb\") " pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.726178 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.726156 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.746048 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.746025 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r747v" Jun 02 19:23:55.880198 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.880167 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-98cf6659c-sfh8t"] Jun 02 19:23:55.882763 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.882735 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r747v"] Jun 02 19:23:55.883915 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:23:55.883839 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d0c8df_f627_41d4_918c_0e60643554ec.slice/crio-6f41e8240f43aef2638d7ad4b983d5a19b2f1eceb146cfee3ab66b95d38e111d WatchSource:0}: Error finding container 6f41e8240f43aef2638d7ad4b983d5a19b2f1eceb146cfee3ab66b95d38e111d: Status 404 returned error can't find the container with id 6f41e8240f43aef2638d7ad4b983d5a19b2f1eceb146cfee3ab66b95d38e111d Jun 02 19:23:55.885824 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:23:55.885792 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6dcacfc_140d_4bed_bb93_18a8436546fb.slice/crio-418c7a7f9cf9e2b2f36729da04d9540cfc4e0c6e7123df2fb5debcdf88240e29 WatchSource:0}: Error finding container 418c7a7f9cf9e2b2f36729da04d9540cfc4e0c6e7123df2fb5debcdf88240e29: Status 404 returned error can't find the container with id 418c7a7f9cf9e2b2f36729da04d9540cfc4e0c6e7123df2fb5debcdf88240e29 Jun 02 19:23:55.969756 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.969659 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r747v" event={"ID":"e6dcacfc-140d-4bed-bb93-18a8436546fb","Type":"ContainerStarted","Data":"eae11321cb3ec80f0bba779b7220e6734ff12f4f14e0a0d61585a5132b49b1f0"} Jun 02 19:23:55.969756 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.969710 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r747v" event={"ID":"e6dcacfc-140d-4bed-bb93-18a8436546fb","Type":"ContainerStarted","Data":"418c7a7f9cf9e2b2f36729da04d9540cfc4e0c6e7123df2fb5debcdf88240e29"} Jun 02 19:23:55.971562 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.971533 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" event={"ID":"93d0c8df-f627-41d4-918c-0e60643554ec","Type":"ContainerStarted","Data":"e430f3c93a4eacd17fd7a6c21045cd6d51c03b95b6c77356abb81684956ce334"} Jun 02 19:23:55.971562 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.971571 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:23:55.971831 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.971584 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" event={"ID":"93d0c8df-f627-41d4-918c-0e60643554ec","Type":"ContainerStarted","Data":"6f41e8240f43aef2638d7ad4b983d5a19b2f1eceb146cfee3ab66b95d38e111d"} Jun 02 19:23:55.991743 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:55.991688 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" podStartSLOduration=0.991673374 podStartE2EDuration="991.673374ms" podCreationTimestamp="2026-06-02 19:23:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:23:55.990803066 +0000 UTC m=+166.080671301" watchObservedRunningTime="2026-06-02 19:23:55.991673374 +0000 UTC m=+166.081541608" Jun 02 19:23:56.389876 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:56.389837 2552 scope.go:117] "RemoveContainer" containerID="1ba4beba848df38468a3aba7d1754217c31e070c0c88092d2742543de0b6cb9b" Jun 02 19:23:56.976127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:56.976024 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r747v" event={"ID":"e6dcacfc-140d-4bed-bb93-18a8436546fb","Type":"ContainerStarted","Data":"45f01fb49b7c1fabbb6a7ee19b7bf71f1192e6a436088844c6cbcb73303066ae"} Jun 02 19:23:56.977938 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:56.977907 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:23:56.978527 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:56.978508 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/1.log" Jun 02 19:23:56.978643 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:56.978544 2552 generic.go:358] "Generic (PLEG): container finished" podID="238f4e41-37e5-454b-84fb-12af646cc790" containerID="f0cd5ad9f107c2db0465da64d699d4cb16b0dc95c01a3a0f9d1316643e09e482" exitCode=255 Jun 02 19:23:56.979428 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:56.979403 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" event={"ID":"238f4e41-37e5-454b-84fb-12af646cc790","Type":"ContainerDied","Data":"f0cd5ad9f107c2db0465da64d699d4cb16b0dc95c01a3a0f9d1316643e09e482"} Jun 02 19:23:56.979535 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:56.979449 2552 scope.go:117] "RemoveContainer" containerID="1ba4beba848df38468a3aba7d1754217c31e070c0c88092d2742543de0b6cb9b" Jun 02 19:23:56.979945 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:56.979924 2552 scope.go:117] "RemoveContainer" containerID="f0cd5ad9f107c2db0465da64d699d4cb16b0dc95c01a3a0f9d1316643e09e482" Jun 02 19:23:56.980221 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:23:56.980198 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f5qhp_openshift-console-operator(238f4e41-37e5-454b-84fb-12af646cc790)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" podUID="238f4e41-37e5-454b-84fb-12af646cc790" Jun 02 19:23:57.983040 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:57.982965 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:23:58.987654 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:23:58.987616 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r747v" event={"ID":"e6dcacfc-140d-4bed-bb93-18a8436546fb","Type":"ContainerStarted","Data":"2176cefcf69be7fe721396061ce365b175c9aa0e54a297c5562635b69d7fd411"} Jun 02 19:24:00.703810 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:00.703765 2552 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:24:00.703810 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:00.703815 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:24:00.704827 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:00.704243 2552 scope.go:117] "RemoveContainer" containerID="f0cd5ad9f107c2db0465da64d699d4cb16b0dc95c01a3a0f9d1316643e09e482" Jun 02 19:24:00.704827 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:24:00.704582 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f5qhp_openshift-console-operator(238f4e41-37e5-454b-84fb-12af646cc790)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" podUID="238f4e41-37e5-454b-84fb-12af646cc790" Jun 02 19:24:00.720658 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:00.720598 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r747v" podStartSLOduration=3.119120906 podStartE2EDuration="5.720582015s" podCreationTimestamp="2026-06-02 19:23:55 +0000 UTC" firstStartedPulling="2026-06-02 19:23:55.94384266 +0000 UTC m=+166.033710873" lastFinishedPulling="2026-06-02 19:23:58.545303757 +0000 UTC m=+168.635171982" observedRunningTime="2026-06-02 19:23:59.009143984 +0000 UTC m=+169.099012208" watchObservedRunningTime="2026-06-02 19:24:00.720582015 +0000 UTC m=+170.810450248" Jun 02 19:24:02.390113 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:02.390068 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:24:02.390113 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:02.390068 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:24:02.393150 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:02.393130 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bv8mp\"" Jun 02 19:24:02.401241 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:02.401202 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zb24h" Jun 02 19:24:02.526225 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:02.526194 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zb24h"] Jun 02 19:24:02.528933 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:02.528893 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb96f33_bcb4_4a7e_b879_159149ced68e.slice/crio-189a7a49e73907632c60813a1b613d7649d3b083ee26303203d8f2e05f183195 WatchSource:0}: Error finding container 189a7a49e73907632c60813a1b613d7649d3b083ee26303203d8f2e05f183195: Status 404 returned error can't find the container with id 189a7a49e73907632c60813a1b613d7649d3b083ee26303203d8f2e05f183195 Jun 02 19:24:03.006797 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:03.006753 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zb24h" event={"ID":"6bb96f33-bcb4-4a7e-b879-159149ced68e","Type":"ContainerStarted","Data":"189a7a49e73907632c60813a1b613d7649d3b083ee26303203d8f2e05f183195"} Jun 02 19:24:04.974289 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:04.974262 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5k45w" Jun 02 19:24:05.013099 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:05.013067 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zb24h" event={"ID":"6bb96f33-bcb4-4a7e-b879-159149ced68e","Type":"ContainerStarted","Data":"17fbe325666515aa4075ef16d547135d4be03de5f6eacd08f7b06f7b0b290abd"} Jun 02 19:24:05.028259 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:05.028194 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zb24h" podStartSLOduration=139.354880027 podStartE2EDuration="2m21.028179073s" podCreationTimestamp="2026-06-02 19:21:44 +0000 UTC" firstStartedPulling="2026-06-02 19:24:02.530693855 +0000 UTC m=+172.620562065" lastFinishedPulling="2026-06-02 19:24:04.203992897 +0000 UTC m=+174.293861111" observedRunningTime="2026-06-02 19:24:05.028018409 +0000 UTC m=+175.117886647" watchObservedRunningTime="2026-06-02 19:24:05.028179073 +0000 UTC m=+175.118047301" Jun 02 19:24:12.509475 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.509438 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-j6zv2"] Jun 02 19:24:12.514167 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.514147 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.516914 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.516894 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 02 19:24:12.517855 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.517832 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 19:24:12.518160 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.517962 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-s8lv6\"" Jun 02 19:24:12.518160 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.517962 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 19:24:12.518160 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.517835 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 02 19:24:12.518160 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.518054 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 19:24:12.520572 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.520553 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-j6zv2"] Jun 02 19:24:12.532874 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.532849 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mlxgv"] Jun 02 19:24:12.535954 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.535933 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.538467 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.538446 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4xbr5\"" Jun 02 19:24:12.538608 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.538547 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 19:24:12.538608 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.538593 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 19:24:12.538842 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.538820 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 19:24:12.541440 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.541415 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbrkn\" (UniqueName: \"kubernetes.io/projected/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-kube-api-access-bbrkn\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.541628 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.541608 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.541767 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.541748 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.541883 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.541866 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.643119 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643082 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-textfile\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.643313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643137 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bbrkn\" (UniqueName: \"kubernetes.io/projected/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-kube-api-access-bbrkn\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.643313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643161 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.643313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643198 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.643313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643219 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.643313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643236 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.643313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643260 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-tls\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.643313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643287 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-sys\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.643313 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643313 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.643749 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643363 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-wtmp\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.643749 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643391 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-root\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.643749 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643417 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23cb6867-6cf0-4299-b352-74534ddc1c83-metrics-client-ca\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.643749 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.643439 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjcnj\" (UniqueName: \"kubernetes.io/projected/23cb6867-6cf0-4299-b352-74534ddc1c83-kube-api-access-wjcnj\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.644149 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.644121 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.645913 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.645884 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.646029 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.645916 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.650768 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.650688 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbrkn\" (UniqueName: \"kubernetes.io/projected/9a361f5f-8ef3-47fb-b76b-4bff82d6c357-kube-api-access-bbrkn\") pod \"openshift-state-metrics-85658796db-j6zv2\" (UID: \"9a361f5f-8ef3-47fb-b76b-4bff82d6c357\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.744692 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.744640 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.744692 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.744695 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-tls\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.744926 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.744723 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-sys\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.744926 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.744747 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.744926 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.744798 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-wtmp\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.744926 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.744828 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-root\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.744926 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.744825 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-sys\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.744926 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.744881 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-root\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.744926 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:24:12.744915 2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 19:24:12.745249 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:24:12.745013 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-tls podName:23cb6867-6cf0-4299-b352-74534ddc1c83 nodeName:}" failed. No retries permitted until 2026-06-02 19:24:13.244990792 +0000 UTC m=+183.334859008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-tls") pod "node-exporter-mlxgv" (UID: "23cb6867-6cf0-4299-b352-74534ddc1c83") : secret "node-exporter-tls" not found Jun 02 19:24:12.745249 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.745018 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-wtmp\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.745249 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.745037 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23cb6867-6cf0-4299-b352-74534ddc1c83-metrics-client-ca\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.745249 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.745071 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wjcnj\" (UniqueName: \"kubernetes.io/projected/23cb6867-6cf0-4299-b352-74534ddc1c83-kube-api-access-wjcnj\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.745249 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.745127 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-textfile\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.745442 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.745278 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.745481 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.745441 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-textfile\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.745632 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.745614 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23cb6867-6cf0-4299-b352-74534ddc1c83-metrics-client-ca\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.747165 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.747147 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.753679 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.753653 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjcnj\" (UniqueName: \"kubernetes.io/projected/23cb6867-6cf0-4299-b352-74534ddc1c83-kube-api-access-wjcnj\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:12.825472 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.825439 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" Jun 02 19:24:12.954993 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:12.954961 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-j6zv2"] Jun 02 19:24:12.958229 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:12.958194 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a361f5f_8ef3_47fb_b76b_4bff82d6c357.slice/crio-cd837793e6e1fe29abfebbd7c528722163f24a5054fa5d7562dceca576d77e73 WatchSource:0}: Error finding container cd837793e6e1fe29abfebbd7c528722163f24a5054fa5d7562dceca576d77e73: Status 404 returned error can't find the container with id cd837793e6e1fe29abfebbd7c528722163f24a5054fa5d7562dceca576d77e73 Jun 02 19:24:13.034761 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.034725 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" event={"ID":"9a361f5f-8ef3-47fb-b76b-4bff82d6c357","Type":"ContainerStarted","Data":"48d2bebe743fa5233d153e52c557d606065a22252a617446445cfb085dac9030"} Jun 02 19:24:13.034845 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.034765 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" event={"ID":"9a361f5f-8ef3-47fb-b76b-4bff82d6c357","Type":"ContainerStarted","Data":"cd837793e6e1fe29abfebbd7c528722163f24a5054fa5d7562dceca576d77e73"} Jun 02 19:24:13.248458 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.248424 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-tls\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:13.250720 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.250690 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23cb6867-6cf0-4299-b352-74534ddc1c83-node-exporter-tls\") pod \"node-exporter-mlxgv\" (UID: \"23cb6867-6cf0-4299-b352-74534ddc1c83\") " pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:13.446963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.446869 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mlxgv" Jun 02 19:24:13.454795 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:13.454762 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cb6867_6cf0_4299_b352_74534ddc1c83.slice/crio-5d09c654122ae5ed4b003d62ebb3674dd5401c436902960011e09ce4c9e3f042 WatchSource:0}: Error finding container 5d09c654122ae5ed4b003d62ebb3674dd5401c436902960011e09ce4c9e3f042: Status 404 returned error can't find the container with id 5d09c654122ae5ed4b003d62ebb3674dd5401c436902960011e09ce4c9e3f042 Jun 02 19:24:13.599695 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.599659 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:24:13.603630 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.603545 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.606556 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.606526 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 19:24:13.606664 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.606561 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 19:24:13.606664 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.606534 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 19:24:13.607044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.606777 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 19:24:13.607044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.606902 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-v558n\"" Jun 02 19:24:13.607044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.606921 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 19:24:13.607044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.606958 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 19:24:13.607044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.607018 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 19:24:13.607365 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.607268 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 19:24:13.613199 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.613177 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 19:24:13.616783 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.616725 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:24:13.650958 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.650925 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9dkt\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-kube-api-access-t9dkt\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651125 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.650973 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651125 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651028 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651125 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651100 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651311 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651147 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651311 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651203 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-out\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651311 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651225 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-web-config\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651311 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651265 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651529 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651323 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651529 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651367 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651529 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651396 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651529 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651432 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.651529 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.651465 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.752852 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.752774 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t9dkt\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-kube-api-access-t9dkt\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.752852 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.752814 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.752852 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.752830 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753119 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.752897 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753119 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.752946 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753119 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753008 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-out\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753119 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753031 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-web-config\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753119 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753080 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753370 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753144 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753370 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753174 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753370 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753201 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753370 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753243 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753370 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753279 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.753983 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.753659 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.754111 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.754078 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.754197 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:24:13.754179 2552 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 02 19:24:13.754252 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:24:13.754242 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls podName:6e63e85f-3e0f-4c91-98c2-36c7a125e7ab nodeName:}" failed. No retries permitted until 2026-06-02 19:24:14.254224223 +0000 UTC m=+184.344092438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab") : secret "alertmanager-main-tls" not found Jun 02 19:24:13.754328 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.754312 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.756546 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.756436 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.757126 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.757102 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-out\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.757469 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.757441 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.757996 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.757716 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.758542 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.758249 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.758542 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.758375 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.758542 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.758473 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.759319 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.759284 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-web-config\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:13.761291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:13.761268 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9dkt\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-kube-api-access-t9dkt\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:14.042473 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:14.041137 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" event={"ID":"9a361f5f-8ef3-47fb-b76b-4bff82d6c357","Type":"ContainerStarted","Data":"4c1a1639f85ea910c8ecdbc0bf04ff84fa3c569a93ac6313e9214d2f9f5a9daf"} Jun 02 19:24:14.042473 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:14.041191 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" event={"ID":"9a361f5f-8ef3-47fb-b76b-4bff82d6c357","Type":"ContainerStarted","Data":"3a14fd69d8bcac09f359b105c05134882d868958e493cc501bee1e283dda3630"} Jun 02 19:24:14.042679 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:14.042589 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlxgv" event={"ID":"23cb6867-6cf0-4299-b352-74534ddc1c83","Type":"ContainerStarted","Data":"5d09c654122ae5ed4b003d62ebb3674dd5401c436902960011e09ce4c9e3f042"} Jun 02 19:24:14.256726 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:14.256685 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:14.259060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:14.259030 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:14.390596 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:14.390552 2552 scope.go:117] "RemoveContainer" containerID="f0cd5ad9f107c2db0465da64d699d4cb16b0dc95c01a3a0f9d1316643e09e482" Jun 02 19:24:14.390816 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:24:14.390796 2552 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f5qhp_openshift-console-operator(238f4e41-37e5-454b-84fb-12af646cc790)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" podUID="238f4e41-37e5-454b-84fb-12af646cc790" Jun 02 19:24:14.515853 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:14.515826 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:24:14.650822 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:14.650796 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:24:14.711372 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:14.711338 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e63e85f_3e0f_4c91_98c2_36c7a125e7ab.slice/crio-65be35c1516fef08646fcecae0eba9b2726cc800069f8179e73f05c70d73887e WatchSource:0}: Error finding container 65be35c1516fef08646fcecae0eba9b2726cc800069f8179e73f05c70d73887e: Status 404 returned error can't find the container with id 65be35c1516fef08646fcecae0eba9b2726cc800069f8179e73f05c70d73887e Jun 02 19:24:15.046366 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.046276 2552 generic.go:358] "Generic (PLEG): container finished" podID="23cb6867-6cf0-4299-b352-74534ddc1c83" containerID="cad93885fc4289b7e669643b050879efb6c13d62d6b099255189ca83ec4595cd" exitCode=0 Jun 02 19:24:15.046366 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.046354 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlxgv" event={"ID":"23cb6867-6cf0-4299-b352-74534ddc1c83","Type":"ContainerDied","Data":"cad93885fc4289b7e669643b050879efb6c13d62d6b099255189ca83ec4595cd"} Jun 02 19:24:15.047519 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.047478 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerStarted","Data":"65be35c1516fef08646fcecae0eba9b2726cc800069f8179e73f05c70d73887e"} Jun 02 19:24:15.081030 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.080981 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-j6zv2" podStartSLOduration=2.202929988 podStartE2EDuration="3.080962385s" podCreationTimestamp="2026-06-02 19:24:12 +0000 UTC" firstStartedPulling="2026-06-02 19:24:13.100907151 +0000 UTC m=+183.190775363" lastFinishedPulling="2026-06-02 19:24:13.978939536 +0000 UTC m=+184.068807760" observedRunningTime="2026-06-02 19:24:15.080644604 +0000 UTC m=+185.170512837" watchObservedRunningTime="2026-06-02 19:24:15.080962385 +0000 UTC m=+185.170830620" Jun 02 19:24:15.530477 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.530440 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-94fcb5df7-285xl"] Jun 02 19:24:15.534040 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.534016 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.536930 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.536898 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 02 19:24:15.537038 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.536934 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 02 19:24:15.537195 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.537180 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 02 19:24:15.537376 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.537362 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xxklk\"" Jun 02 19:24:15.537590 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.537543 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 02 19:24:15.537590 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.537607 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 02 19:24:15.537930 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.537864 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bq2esnlivht0g\"" Jun 02 19:24:15.553372 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.553339 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-94fcb5df7-285xl"] Jun 02 19:24:15.568763 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.568733 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.568898 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.568785 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4889ba7-21a1-41ca-b62b-9432f10bb11f-metrics-client-ca\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.568898 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.568839 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-tls\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.569005 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.568911 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.569005 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.568936 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckngq\" (UniqueName: \"kubernetes.io/projected/b4889ba7-21a1-41ca-b62b-9432f10bb11f-kube-api-access-ckngq\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.569005 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.568991 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.569118 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.569031 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-grpc-tls\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.569118 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.569074 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.670140 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.670107 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.670140 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.670144 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ckngq\" (UniqueName: \"kubernetes.io/projected/b4889ba7-21a1-41ca-b62b-9432f10bb11f-kube-api-access-ckngq\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.670684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.670194 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.670684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.670224 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-grpc-tls\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.670684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.670263 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.670684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.670295 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.670684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.670330 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4889ba7-21a1-41ca-b62b-9432f10bb11f-metrics-client-ca\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.670684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.670367 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-tls\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.671172 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.671119 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4889ba7-21a1-41ca-b62b-9432f10bb11f-metrics-client-ca\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.673807 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.673778 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.674022 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.673998 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.674079 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.674019 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.674526 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.674481 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.674612 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.674530 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-thanos-querier-tls\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.675518 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.675482 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4889ba7-21a1-41ca-b62b-9432f10bb11f-secret-grpc-tls\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.684198 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.684174 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckngq\" (UniqueName: \"kubernetes.io/projected/b4889ba7-21a1-41ca-b62b-9432f10bb11f-kube-api-access-ckngq\") pod \"thanos-querier-94fcb5df7-285xl\" (UID: \"b4889ba7-21a1-41ca-b62b-9432f10bb11f\") " pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.846940 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.846903 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:15.996162 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:15.996137 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-94fcb5df7-285xl"] Jun 02 19:24:15.999526 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:15.999449 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4889ba7_21a1_41ca_b62b_9432f10bb11f.slice/crio-bbf29a81c702de5bac3fb314b314c09db80809e7e6183463399ae19bc42790fe WatchSource:0}: Error finding container bbf29a81c702de5bac3fb314b314c09db80809e7e6183463399ae19bc42790fe: Status 404 returned error can't find the container with id bbf29a81c702de5bac3fb314b314c09db80809e7e6183463399ae19bc42790fe Jun 02 19:24:16.052133 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.052103 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlxgv" event={"ID":"23cb6867-6cf0-4299-b352-74534ddc1c83","Type":"ContainerStarted","Data":"d0ea1ff3d299b2c9f941e8c26ff2b8d2828c48dd161a4fc7b27a9e3fe23c1c3b"} Jun 02 19:24:16.052261 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.052139 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlxgv" event={"ID":"23cb6867-6cf0-4299-b352-74534ddc1c83","Type":"ContainerStarted","Data":"e01aa7ed41763e0e584295e8ae3ff88220ec6c565775fef7f225a9c8ec8b9ef9"} Jun 02 19:24:16.053433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.053407 2552 generic.go:358] "Generic (PLEG): container finished" podID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerID="56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6" exitCode=0 Jun 02 19:24:16.053547 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.053488 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerDied","Data":"56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6"} Jun 02 19:24:16.059134 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.059110 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" event={"ID":"b4889ba7-21a1-41ca-b62b-9432f10bb11f","Type":"ContainerStarted","Data":"bbf29a81c702de5bac3fb314b314c09db80809e7e6183463399ae19bc42790fe"} Jun 02 19:24:16.075231 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.075187 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mlxgv" podStartSLOduration=3.026704104 podStartE2EDuration="4.075175712s" podCreationTimestamp="2026-06-02 19:24:12 +0000 UTC" firstStartedPulling="2026-06-02 19:24:13.456364401 +0000 UTC m=+183.546232612" lastFinishedPulling="2026-06-02 19:24:14.504836006 +0000 UTC m=+184.594704220" observedRunningTime="2026-06-02 19:24:16.07440992 +0000 UTC m=+186.164278153" watchObservedRunningTime="2026-06-02 19:24:16.075175712 +0000 UTC m=+186.165043945" Jun 02 19:24:16.940788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.940747 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78c7859fc5-dsfjk"] Jun 02 19:24:16.944426 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.944396 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:16.948276 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.948248 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 02 19:24:16.948427 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.948300 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dmuco9lt55a0s\"" Jun 02 19:24:16.948427 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.948311 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q6msl\"" Jun 02 19:24:16.948427 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.948253 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 19:24:16.948427 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.948259 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 02 19:24:16.948700 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.948248 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 02 19:24:16.955861 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.955817 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78c7859fc5-dsfjk"] Jun 02 19:24:16.982791 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.982751 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/180b464f-74a6-41e3-8d53-89f87b51f55a-metrics-server-audit-profiles\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:16.983026 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.982808 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-secret-metrics-server-tls\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:16.983026 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.982837 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/180b464f-74a6-41e3-8d53-89f87b51f55a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:16.983026 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.982987 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-secret-metrics-server-client-certs\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:16.983188 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.983027 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/180b464f-74a6-41e3-8d53-89f87b51f55a-audit-log\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:16.983188 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.983068 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt9vd\" (UniqueName: \"kubernetes.io/projected/180b464f-74a6-41e3-8d53-89f87b51f55a-kube-api-access-rt9vd\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:16.983188 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.983119 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-client-ca-bundle\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:16.986512 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:16.986011 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-98cf6659c-sfh8t" Jun 02 19:24:17.084471 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.084417 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/180b464f-74a6-41e3-8d53-89f87b51f55a-metrics-server-audit-profiles\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.084707 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.084526 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-secret-metrics-server-tls\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.084707 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.084556 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/180b464f-74a6-41e3-8d53-89f87b51f55a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.084707 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.084610 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-secret-metrics-server-client-certs\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.084707 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.084639 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/180b464f-74a6-41e3-8d53-89f87b51f55a-audit-log\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.084707 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.084701 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rt9vd\" (UniqueName: \"kubernetes.io/projected/180b464f-74a6-41e3-8d53-89f87b51f55a-kube-api-access-rt9vd\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.085057 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.084806 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-client-ca-bundle\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.085696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.085438 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/180b464f-74a6-41e3-8d53-89f87b51f55a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.085696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.085631 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/180b464f-74a6-41e3-8d53-89f87b51f55a-metrics-server-audit-profiles\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.085851 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.085759 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/180b464f-74a6-41e3-8d53-89f87b51f55a-audit-log\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.087833 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.087807 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-secret-metrics-server-client-certs\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.090231 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.090184 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-client-ca-bundle\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.090352 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.090232 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/180b464f-74a6-41e3-8d53-89f87b51f55a-secret-metrics-server-tls\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.096865 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.096833 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt9vd\" (UniqueName: \"kubernetes.io/projected/180b464f-74a6-41e3-8d53-89f87b51f55a-kube-api-access-rt9vd\") pod \"metrics-server-78c7859fc5-dsfjk\" (UID: \"180b464f-74a6-41e3-8d53-89f87b51f55a\") " pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.257034 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.256945 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:17.775394 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.775363 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-b6488f97-mkgxp"] Jun 02 19:24:17.779282 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.779259 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.780766 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.780710 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78c7859fc5-dsfjk"] Jun 02 19:24:17.782178 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.781976 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 02 19:24:17.782245 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.782186 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-z8gvs\"" Jun 02 19:24:17.782425 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.782409 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 02 19:24:17.783653 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.783635 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 02 19:24:17.783986 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.783963 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 02 19:24:17.784994 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.784771 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 02 19:24:17.786139 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:17.786090 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180b464f_74a6_41e3_8d53_89f87b51f55a.slice/crio-c184990fe9647da1f07ecf234ae798f22fc8a09e58dd986cfcdcb7456d30a5f5 WatchSource:0}: Error finding container c184990fe9647da1f07ecf234ae798f22fc8a09e58dd986cfcdcb7456d30a5f5: Status 404 returned error can't find the container with id c184990fe9647da1f07ecf234ae798f22fc8a09e58dd986cfcdcb7456d30a5f5 Jun 02 19:24:17.794016 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.793993 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 02 19:24:17.801518 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.800226 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b6488f97-mkgxp"] Jun 02 19:24:17.892634 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.892540 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-telemeter-client-tls\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.892790 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.892663 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-serving-certs-ca-bundle\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.892790 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.892753 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-secret-telemeter-client\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.892790 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.892785 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.892927 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.892855 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.892927 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.892908 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-metrics-client-ca\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.893023 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.892939 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfr7t\" (UniqueName: \"kubernetes.io/projected/7f511b0a-7259-4f91-88f4-ec4b262067e7-kube-api-access-jfr7t\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.893063 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.893033 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-federate-client-tls\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.994403 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.994362 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-serving-certs-ca-bundle\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.994909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.994420 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-secret-telemeter-client\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.994909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.994447 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.994909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.994481 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.994909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.994533 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-metrics-client-ca\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.994909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.994566 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jfr7t\" (UniqueName: \"kubernetes.io/projected/7f511b0a-7259-4f91-88f4-ec4b262067e7-kube-api-access-jfr7t\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.994909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.994642 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-federate-client-tls\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.994909 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.994685 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-telemeter-client-tls\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.995306 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.995235 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-serving-certs-ca-bundle\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.995669 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.995636 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.995778 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.995750 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f511b0a-7259-4f91-88f4-ec4b262067e7-metrics-client-ca\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.997922 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.997886 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-telemeter-client-tls\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.998050 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.998027 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.998107 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.998074 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-secret-telemeter-client\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:17.998604 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:17.998580 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f511b0a-7259-4f91-88f4-ec4b262067e7-federate-client-tls\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:18.003422 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:18.003399 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfr7t\" (UniqueName: \"kubernetes.io/projected/7f511b0a-7259-4f91-88f4-ec4b262067e7-kube-api-access-jfr7t\") pod \"telemeter-client-b6488f97-mkgxp\" (UID: \"7f511b0a-7259-4f91-88f4-ec4b262067e7\") " pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:18.073095 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:18.073052 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" event={"ID":"180b464f-74a6-41e3-8d53-89f87b51f55a","Type":"ContainerStarted","Data":"c184990fe9647da1f07ecf234ae798f22fc8a09e58dd986cfcdcb7456d30a5f5"} Jun 02 19:24:18.074977 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:18.074951 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerStarted","Data":"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1"} Jun 02 19:24:18.093257 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:18.093220 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" Jun 02 19:24:18.476137 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:18.476081 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b6488f97-mkgxp"] Jun 02 19:24:18.482833 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:18.482798 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f511b0a_7259_4f91_88f4_ec4b262067e7.slice/crio-45597b6c4463222ef6749ce15728ab4bd200e99957bb04a219b6373d6d64c95c WatchSource:0}: Error finding container 45597b6c4463222ef6749ce15728ab4bd200e99957bb04a219b6373d6d64c95c: Status 404 returned error can't find the container with id 45597b6c4463222ef6749ce15728ab4bd200e99957bb04a219b6373d6d64c95c Jun 02 19:24:19.083770 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:19.083727 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" event={"ID":"b4889ba7-21a1-41ca-b62b-9432f10bb11f","Type":"ContainerStarted","Data":"8d0889b1a3463ecbd5f3046e678f095aa3a72551f25494f7e072d0466963f6f1"} Jun 02 19:24:19.084243 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:19.083778 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" event={"ID":"b4889ba7-21a1-41ca-b62b-9432f10bb11f","Type":"ContainerStarted","Data":"16e1ab44d837aa7e4004dffc147d6352c94fc917a38c019ef70639caa919395c"} Jun 02 19:24:19.084243 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:19.083795 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" event={"ID":"b4889ba7-21a1-41ca-b62b-9432f10bb11f","Type":"ContainerStarted","Data":"c284cfc65111d6742fe706108c71528505c08ed9ea3879cbe0014fd71283af5f"} Jun 02 19:24:19.088152 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:19.088121 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerStarted","Data":"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4"} Jun 02 19:24:19.088307 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:19.088161 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerStarted","Data":"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6"} Jun 02 19:24:19.088307 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:19.088176 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerStarted","Data":"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71"} Jun 02 19:24:19.088307 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:19.088189 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerStarted","Data":"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746"} Jun 02 19:24:19.089718 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:19.089683 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" event={"ID":"7f511b0a-7259-4f91-88f4-ec4b262067e7","Type":"ContainerStarted","Data":"45597b6c4463222ef6749ce15728ab4bd200e99957bb04a219b6373d6d64c95c"} Jun 02 19:24:20.095212 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.095174 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" event={"ID":"180b464f-74a6-41e3-8d53-89f87b51f55a","Type":"ContainerStarted","Data":"ecb5d5825e15c3e84de3a111f2d0a7a40050104629786d5753819f87c09b05ab"} Jun 02 19:24:20.099952 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.098998 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerStarted","Data":"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c"} Jun 02 19:24:20.103162 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.103136 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" event={"ID":"b4889ba7-21a1-41ca-b62b-9432f10bb11f","Type":"ContainerStarted","Data":"fc38f7373f1ccba171c2ed504d08d7d40cf3edcd6178d4af0a81af8734d6ba96"} Jun 02 19:24:20.103162 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.103163 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" event={"ID":"b4889ba7-21a1-41ca-b62b-9432f10bb11f","Type":"ContainerStarted","Data":"6ce3449bf9affaf762349f79d8c44436186b762720a0bdc7dc60e9606571b5b4"} Jun 02 19:24:20.103330 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.103173 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" event={"ID":"b4889ba7-21a1-41ca-b62b-9432f10bb11f","Type":"ContainerStarted","Data":"74797d3b12528aa0feb976ef9697cae180732282d1e9e3656d8070ad11dacf49"} Jun 02 19:24:20.103330 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.103302 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:20.117165 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.117103 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" podStartSLOduration=2.110263843 podStartE2EDuration="4.117086695s" podCreationTimestamp="2026-06-02 19:24:16 +0000 UTC" firstStartedPulling="2026-06-02 19:24:17.791021942 +0000 UTC m=+187.880890155" lastFinishedPulling="2026-06-02 19:24:19.797844791 +0000 UTC m=+189.887713007" observedRunningTime="2026-06-02 19:24:20.116048143 +0000 UTC m=+190.205916380" watchObservedRunningTime="2026-06-02 19:24:20.117086695 +0000 UTC m=+190.206954930" Jun 02 19:24:20.141185 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.141129 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" podStartSLOduration=1.343249135 podStartE2EDuration="5.141110231s" podCreationTimestamp="2026-06-02 19:24:15 +0000 UTC" firstStartedPulling="2026-06-02 19:24:16.001772699 +0000 UTC m=+186.091640914" lastFinishedPulling="2026-06-02 19:24:19.799633788 +0000 UTC m=+189.889502010" observedRunningTime="2026-06-02 19:24:20.140101944 +0000 UTC m=+190.229970177" watchObservedRunningTime="2026-06-02 19:24:20.141110231 +0000 UTC m=+190.230978543" Jun 02 19:24:20.177439 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:20.177381 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.093048181 podStartE2EDuration="7.177364789s" podCreationTimestamp="2026-06-02 19:24:13 +0000 UTC" firstStartedPulling="2026-06-02 19:24:14.713296575 +0000 UTC m=+184.803164789" lastFinishedPulling="2026-06-02 19:24:19.797613173 +0000 UTC m=+189.887481397" observedRunningTime="2026-06-02 19:24:20.175814981 +0000 UTC m=+190.265683214" watchObservedRunningTime="2026-06-02 19:24:20.177364789 +0000 UTC m=+190.267233063" Jun 02 19:24:21.108472 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:21.108429 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" event={"ID":"7f511b0a-7259-4f91-88f4-ec4b262067e7","Type":"ContainerStarted","Data":"11971c9aa6014e6d2a0fe5fb59272e0a7e890908276cb10186df3f66224f73d5"} Jun 02 19:24:21.108472 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:21.108474 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" event={"ID":"7f511b0a-7259-4f91-88f4-ec4b262067e7","Type":"ContainerStarted","Data":"2c2035ff9ebc393c4dc9b23934f5f2eecb531cdd5c1a3665b57f4587f662be56"} Jun 02 19:24:21.109026 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:21.108488 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" event={"ID":"7f511b0a-7259-4f91-88f4-ec4b262067e7","Type":"ContainerStarted","Data":"0a9854cb59d981d25c9bd64a2453c738b8ae1e65065f44ad4679bec3d3aef9c4"} Jun 02 19:24:21.141696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:21.141642 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-b6488f97-mkgxp" podStartSLOduration=1.8567661640000002 podStartE2EDuration="4.141625541s" podCreationTimestamp="2026-06-02 19:24:17 +0000 UTC" firstStartedPulling="2026-06-02 19:24:18.485024301 +0000 UTC m=+188.574892531" lastFinishedPulling="2026-06-02 19:24:20.769883683 +0000 UTC m=+190.859751908" observedRunningTime="2026-06-02 19:24:21.140719056 +0000 UTC m=+191.230587291" watchObservedRunningTime="2026-06-02 19:24:21.141625541 +0000 UTC m=+191.231493785" Jun 02 19:24:26.114956 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:26.114925 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-94fcb5df7-285xl" Jun 02 19:24:26.390184 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:26.390082 2552 scope.go:117] "RemoveContainer" containerID="f0cd5ad9f107c2db0465da64d699d4cb16b0dc95c01a3a0f9d1316643e09e482" Jun 02 19:24:27.119324 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.119286 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-pgl2t"] Jun 02 19:24:27.122857 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.122833 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-pgl2t" Jun 02 19:24:27.125163 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.125132 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 19:24:27.125316 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.125191 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-85ktl\"" Jun 02 19:24:27.125316 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.125223 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 19:24:27.127704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.127681 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:24:27.127839 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.127786 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" event={"ID":"238f4e41-37e5-454b-84fb-12af646cc790","Type":"ContainerStarted","Data":"71fb634cf15fc30ddfeac79cc3802c93ed441f5f91327910ef91120f28033e91"} Jun 02 19:24:27.128067 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.128051 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:24:27.132439 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.132412 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-pgl2t"] Jun 02 19:24:27.133897 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.133875 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" Jun 02 19:24:27.157250 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.157189 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-f5qhp" podStartSLOduration=54.621722937 podStartE2EDuration="57.157172843s" podCreationTimestamp="2026-06-02 19:23:30 +0000 UTC" firstStartedPulling="2026-06-02 19:23:30.85128005 +0000 UTC m=+140.941148261" lastFinishedPulling="2026-06-02 19:23:33.386729942 +0000 UTC m=+143.476598167" observedRunningTime="2026-06-02 19:24:27.156325012 +0000 UTC m=+197.246193246" watchObservedRunningTime="2026-06-02 19:24:27.157172843 +0000 UTC m=+197.247041098" Jun 02 19:24:27.182896 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.182857 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hbzg\" (UniqueName: \"kubernetes.io/projected/55a7d134-12a2-4bd6-8c88-4dd937c306c7-kube-api-access-4hbzg\") pod \"downloads-c65b8dc4b-pgl2t\" (UID: \"55a7d134-12a2-4bd6-8c88-4dd937c306c7\") " pod="openshift-console/downloads-c65b8dc4b-pgl2t" Jun 02 19:24:27.284000 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.283949 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4hbzg\" (UniqueName: \"kubernetes.io/projected/55a7d134-12a2-4bd6-8c88-4dd937c306c7-kube-api-access-4hbzg\") pod \"downloads-c65b8dc4b-pgl2t\" (UID: \"55a7d134-12a2-4bd6-8c88-4dd937c306c7\") " pod="openshift-console/downloads-c65b8dc4b-pgl2t" Jun 02 19:24:27.293477 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.293444 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hbzg\" (UniqueName: \"kubernetes.io/projected/55a7d134-12a2-4bd6-8c88-4dd937c306c7-kube-api-access-4hbzg\") pod \"downloads-c65b8dc4b-pgl2t\" (UID: \"55a7d134-12a2-4bd6-8c88-4dd937c306c7\") " pod="openshift-console/downloads-c65b8dc4b-pgl2t" Jun 02 19:24:27.434789 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.434693 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-pgl2t" Jun 02 19:24:27.561514 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:27.561467 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-pgl2t"] Jun 02 19:24:27.565285 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:27.565255 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a7d134_12a2_4bd6_8c88_4dd937c306c7.slice/crio-8522c93cd28fb386d9c1496e7d5edc27a9dbbd19c3b1738bbda72e46d4b20a84 WatchSource:0}: Error finding container 8522c93cd28fb386d9c1496e7d5edc27a9dbbd19c3b1738bbda72e46d4b20a84: Status 404 returned error can't find the container with id 8522c93cd28fb386d9c1496e7d5edc27a9dbbd19c3b1738bbda72e46d4b20a84 Jun 02 19:24:28.132872 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:28.132810 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-pgl2t" event={"ID":"55a7d134-12a2-4bd6-8c88-4dd937c306c7","Type":"ContainerStarted","Data":"8522c93cd28fb386d9c1496e7d5edc27a9dbbd19c3b1738bbda72e46d4b20a84"} Jun 02 19:24:32.711100 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.711065 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-ddb8cfb64-8px6n"] Jun 02 19:24:32.719397 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.719373 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.722750 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.722721 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-npqlq\"" Jun 02 19:24:32.724466 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.723786 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 19:24:32.724466 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.723858 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 19:24:32.724466 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.724003 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 19:24:32.724466 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.724074 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 19:24:32.724466 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.724014 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 19:24:32.726273 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.726235 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ddb8cfb64-8px6n"] Jun 02 19:24:32.835426 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.835389 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-oauth-serving-cert\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.835426 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.835433 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-oauth-config\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.835672 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.835472 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-service-ca\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.835672 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.835535 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjgq2\" (UniqueName: \"kubernetes.io/projected/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-kube-api-access-qjgq2\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.835672 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.835581 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-serving-cert\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.835672 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.835640 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-config\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.936557 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.936517 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-service-ca\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.936749 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.936576 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qjgq2\" (UniqueName: \"kubernetes.io/projected/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-kube-api-access-qjgq2\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.936749 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.936632 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-serving-cert\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.936875 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.936744 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-config\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.937087 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.937010 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-oauth-serving-cert\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.937234 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.937211 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-service-ca\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.937316 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.937217 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-oauth-config\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.937672 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.937618 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-config\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.937795 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.937770 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-oauth-serving-cert\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.939732 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.939709 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-serving-cert\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.939873 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.939853 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-oauth-config\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:32.947441 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:32.947396 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjgq2\" (UniqueName: \"kubernetes.io/projected/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-kube-api-access-qjgq2\") pod \"console-ddb8cfb64-8px6n\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:33.033415 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:33.033325 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:33.187736 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:33.187580 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ddb8cfb64-8px6n"] Jun 02 19:24:33.191105 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:33.191070 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e36f3d4_d8bf_4cd6_8575_bcb6f836959f.slice/crio-724c232bf95727179a06c19ef80db52572acc137c1f9d205610c28b2546764ee WatchSource:0}: Error finding container 724c232bf95727179a06c19ef80db52572acc137c1f9d205610c28b2546764ee: Status 404 returned error can't find the container with id 724c232bf95727179a06c19ef80db52572acc137c1f9d205610c28b2546764ee Jun 02 19:24:34.154768 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:34.154724 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddb8cfb64-8px6n" event={"ID":"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f","Type":"ContainerStarted","Data":"724c232bf95727179a06c19ef80db52572acc137c1f9d205610c28b2546764ee"} Jun 02 19:24:37.166966 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:37.166926 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddb8cfb64-8px6n" event={"ID":"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f","Type":"ContainerStarted","Data":"6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6"} Jun 02 19:24:37.187838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:37.187719 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-ddb8cfb64-8px6n" podStartSLOduration=1.445096157 podStartE2EDuration="5.187696749s" podCreationTimestamp="2026-06-02 19:24:32 +0000 UTC" firstStartedPulling="2026-06-02 19:24:33.193523415 +0000 UTC m=+203.283391627" lastFinishedPulling="2026-06-02 19:24:36.936123995 +0000 UTC m=+207.025992219" observedRunningTime="2026-06-02 19:24:37.184328028 +0000 UTC m=+207.274196265" watchObservedRunningTime="2026-06-02 19:24:37.187696749 +0000 UTC m=+207.277564983" Jun 02 19:24:37.257866 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:37.257823 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:37.258056 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:37.257914 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:42.622026 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.621994 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-d78699465-mm9ch"] Jun 02 19:24:42.625049 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.625023 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.631693 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.631667 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-serving-cert\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.631816 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.631723 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-service-ca\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.631816 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.631765 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8hk2\" (UniqueName: \"kubernetes.io/projected/5d835815-c2ab-4c91-bd53-e991b93314c9-kube-api-access-r8hk2\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.631939 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.631815 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-console-config\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.631939 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.631901 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-oauth-config\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.632045 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.631951 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-trusted-ca-bundle\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.632045 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.632004 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-oauth-serving-cert\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.633383 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.633361 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 02 19:24:42.636254 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.636230 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d78699465-mm9ch"] Jun 02 19:24:42.733232 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.733171 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r8hk2\" (UniqueName: \"kubernetes.io/projected/5d835815-c2ab-4c91-bd53-e991b93314c9-kube-api-access-r8hk2\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.733426 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.733239 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-console-config\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.733426 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.733305 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-oauth-config\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.733426 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.733346 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-trusted-ca-bundle\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.733426 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.733390 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-oauth-serving-cert\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.733426 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.733426 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-serving-cert\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.733686 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.733465 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-service-ca\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.734128 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.734075 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-console-config\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.734128 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.734097 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-service-ca\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.734323 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.734250 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-oauth-serving-cert\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.734559 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.734529 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-trusted-ca-bundle\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.736325 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.736303 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-oauth-config\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.736770 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.736746 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-serving-cert\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.742897 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.742877 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8hk2\" (UniqueName: \"kubernetes.io/projected/5d835815-c2ab-4c91-bd53-e991b93314c9-kube-api-access-r8hk2\") pod \"console-d78699465-mm9ch\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:42.935724 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:42.935627 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:43.034098 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:43.034063 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:43.034274 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:43.034114 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:43.039979 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:43.039952 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:43.190864 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:43.190784 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:24:44.941092 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:44.941071 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d78699465-mm9ch"] Jun 02 19:24:44.942633 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:24:44.942611 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d835815_c2ab_4c91_bd53_e991b93314c9.slice/crio-76ae8aa83a9da05455ddddfe2df9397c750d1454b9e2682653f1a718edcc01ab WatchSource:0}: Error finding container 76ae8aa83a9da05455ddddfe2df9397c750d1454b9e2682653f1a718edcc01ab: Status 404 returned error can't find the container with id 76ae8aa83a9da05455ddddfe2df9397c750d1454b9e2682653f1a718edcc01ab Jun 02 19:24:45.195632 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:45.195410 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-pgl2t" event={"ID":"55a7d134-12a2-4bd6-8c88-4dd937c306c7","Type":"ContainerStarted","Data":"690f6d0e0ccdcab8967739942ff043c332a903ea6df574508edf11b4cececcad"} Jun 02 19:24:45.195632 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:45.195600 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-pgl2t" Jun 02 19:24:45.197048 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:45.197018 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d78699465-mm9ch" event={"ID":"5d835815-c2ab-4c91-bd53-e991b93314c9","Type":"ContainerStarted","Data":"150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4"} Jun 02 19:24:45.197188 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:45.197052 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d78699465-mm9ch" event={"ID":"5d835815-c2ab-4c91-bd53-e991b93314c9","Type":"ContainerStarted","Data":"76ae8aa83a9da05455ddddfe2df9397c750d1454b9e2682653f1a718edcc01ab"} Jun 02 19:24:45.210608 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:45.210577 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-pgl2t" Jun 02 19:24:45.216032 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:45.215980 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-pgl2t" podStartSLOduration=0.866235192 podStartE2EDuration="18.21596333s" podCreationTimestamp="2026-06-02 19:24:27 +0000 UTC" firstStartedPulling="2026-06-02 19:24:27.567163626 +0000 UTC m=+197.657031837" lastFinishedPulling="2026-06-02 19:24:44.916891755 +0000 UTC m=+215.006759975" observedRunningTime="2026-06-02 19:24:45.21452131 +0000 UTC m=+215.304389546" watchObservedRunningTime="2026-06-02 19:24:45.21596333 +0000 UTC m=+215.305831564" Jun 02 19:24:45.253804 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:45.253736 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-d78699465-mm9ch" podStartSLOduration=3.253716537 podStartE2EDuration="3.253716537s" podCreationTimestamp="2026-06-02 19:24:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:24:45.252710991 +0000 UTC m=+215.342579225" watchObservedRunningTime="2026-06-02 19:24:45.253716537 +0000 UTC m=+215.343584769" Jun 02 19:24:52.935797 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:52.935751 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:52.936363 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:52.935822 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:52.941820 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:52.941792 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:53.231348 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:53.231265 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:24:53.287130 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:53.287091 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ddb8cfb64-8px6n"] Jun 02 19:24:55.234688 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:55.234654 2552 generic.go:358] "Generic (PLEG): container finished" podID="e563216c-9d47-4016-b694-2fc740bccd5d" containerID="f3ab26f0fbaf88ed62cf361d992b4571f38f5b754a2b91345dd25adf41bde1f2" exitCode=0 Jun 02 19:24:55.235070 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:55.234726 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" event={"ID":"e563216c-9d47-4016-b694-2fc740bccd5d","Type":"ContainerDied","Data":"f3ab26f0fbaf88ed62cf361d992b4571f38f5b754a2b91345dd25adf41bde1f2"} Jun 02 19:24:55.235115 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:55.235105 2552 scope.go:117] "RemoveContainer" containerID="f3ab26f0fbaf88ed62cf361d992b4571f38f5b754a2b91345dd25adf41bde1f2" Jun 02 19:24:56.239381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:56.239346 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-fk87w" event={"ID":"e563216c-9d47-4016-b694-2fc740bccd5d","Type":"ContainerStarted","Data":"fb7a6d0c46de74eddd4b7de7a5c865af91f915f99576c47d8cd3308cf13fa759"} Jun 02 19:24:57.262708 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:57.262680 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:24:57.267019 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:24:57.266994 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78c7859fc5-dsfjk" Jun 02 19:25:18.317402 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.317329 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-ddb8cfb64-8px6n" podUID="2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" containerName="console" containerID="cri-o://6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6" gracePeriod=15 Jun 02 19:25:18.592261 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.592235 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-ddb8cfb64-8px6n_2e36f3d4-d8bf-4cd6-8575-bcb6f836959f/console/0.log" Jun 02 19:25:18.592400 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.592316 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:25:18.689060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689019 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-oauth-config\") pod \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " Jun 02 19:25:18.689060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689067 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-serving-cert\") pod \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " Jun 02 19:25:18.689322 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689108 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-config\") pod \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " Jun 02 19:25:18.689322 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689280 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-oauth-serving-cert\") pod \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " Jun 02 19:25:18.689430 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689353 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjgq2\" (UniqueName: \"kubernetes.io/projected/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-kube-api-access-qjgq2\") pod \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " Jun 02 19:25:18.689430 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689424 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-service-ca\") pod \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\" (UID: \"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f\") " Jun 02 19:25:18.689583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689470 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-config" (OuterVolumeSpecName: "console-config") pod "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" (UID: "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:25:18.689713 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689684 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" (UID: "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:25:18.689778 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689760 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-config\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:18.689778 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689759 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-service-ca" (OuterVolumeSpecName: "service-ca") pod "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" (UID: "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:25:18.689882 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.689778 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-oauth-serving-cert\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:18.691456 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.691427 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" (UID: "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:18.691606 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.691470 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" (UID: "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:18.691606 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.691506 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-kube-api-access-qjgq2" (OuterVolumeSpecName: "kube-api-access-qjgq2") pod "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" (UID: "2e36f3d4-d8bf-4cd6-8575-bcb6f836959f"). InnerVolumeSpecName "kube-api-access-qjgq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:25:18.790446 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.790402 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjgq2\" (UniqueName: \"kubernetes.io/projected/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-kube-api-access-qjgq2\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:18.790446 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.790440 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-service-ca\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:18.790446 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.790452 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-oauth-config\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:18.790710 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:18.790462 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f-console-serving-cert\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:19.310746 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.310717 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-ddb8cfb64-8px6n_2e36f3d4-d8bf-4cd6-8575-bcb6f836959f/console/0.log" Jun 02 19:25:19.310935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.310757 2552 generic.go:358] "Generic (PLEG): container finished" podID="2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" containerID="6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6" exitCode=2 Jun 02 19:25:19.310935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.310826 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddb8cfb64-8px6n" Jun 02 19:25:19.310935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.310839 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddb8cfb64-8px6n" event={"ID":"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f","Type":"ContainerDied","Data":"6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6"} Jun 02 19:25:19.310935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.310867 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddb8cfb64-8px6n" event={"ID":"2e36f3d4-d8bf-4cd6-8575-bcb6f836959f","Type":"ContainerDied","Data":"724c232bf95727179a06c19ef80db52572acc137c1f9d205610c28b2546764ee"} Jun 02 19:25:19.310935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.310881 2552 scope.go:117] "RemoveContainer" containerID="6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6" Jun 02 19:25:19.319686 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.319353 2552 scope.go:117] "RemoveContainer" containerID="6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6" Jun 02 19:25:19.319686 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:25:19.319646 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6\": container with ID starting with 6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6 not found: ID does not exist" containerID="6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6" Jun 02 19:25:19.319928 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.319679 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6"} err="failed to get container status \"6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6\": rpc error: code = NotFound desc = could not find container \"6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6\": container with ID starting with 6c4e62f959638425203fb11c0b102f48bad3e04df6b109a7c2e59d060ddf89b6 not found: ID does not exist" Jun 02 19:25:19.335937 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.335910 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ddb8cfb64-8px6n"] Jun 02 19:25:19.340180 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:19.340144 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-ddb8cfb64-8px6n"] Jun 02 19:25:20.396774 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:20.394906 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" path="/var/lib/kubelet/pods/2e36f3d4-d8bf-4cd6-8575-bcb6f836959f/volumes" Jun 02 19:25:22.223023 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:22.222975 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:25:22.225512 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:22.225476 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/386d5a57-cdc7-4d70-b3af-c06ea739c213-metrics-certs\") pod \"network-metrics-daemon-sgj2b\" (UID: \"386d5a57-cdc7-4d70-b3af-c06ea739c213\") " pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:25:22.493729 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:22.493657 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tb5vh\"" Jun 02 19:25:22.501743 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:22.501724 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgj2b" Jun 02 19:25:22.623728 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:22.623704 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sgj2b"] Jun 02 19:25:22.626258 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:25:22.626227 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod386d5a57_cdc7_4d70_b3af_c06ea739c213.slice/crio-c7ad611cea9fad969b3cbc6293552e94c51ef16aa5524aca824da8b4c0fbfe0b WatchSource:0}: Error finding container c7ad611cea9fad969b3cbc6293552e94c51ef16aa5524aca824da8b4c0fbfe0b: Status 404 returned error can't find the container with id c7ad611cea9fad969b3cbc6293552e94c51ef16aa5524aca824da8b4c0fbfe0b Jun 02 19:25:23.324608 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:23.324568 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgj2b" event={"ID":"386d5a57-cdc7-4d70-b3af-c06ea739c213","Type":"ContainerStarted","Data":"c7ad611cea9fad969b3cbc6293552e94c51ef16aa5524aca824da8b4c0fbfe0b"} Jun 02 19:25:25.332485 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:25.332446 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgj2b" event={"ID":"386d5a57-cdc7-4d70-b3af-c06ea739c213","Type":"ContainerStarted","Data":"969e73118198e35ae27f01e28e62f3b843b3cae115c50e7cc080be437af87614"} Jun 02 19:25:25.332485 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:25.332488 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgj2b" event={"ID":"386d5a57-cdc7-4d70-b3af-c06ea739c213","Type":"ContainerStarted","Data":"d9e5453bf84ca604d502778bce4758b5b2c21aaa7191e5a86b77aac35b69dc87"} Jun 02 19:25:25.352230 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:25.352183 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sgj2b" podStartSLOduration=253.614868214 podStartE2EDuration="4m15.352169487s" podCreationTimestamp="2026-06-02 19:21:10 +0000 UTC" firstStartedPulling="2026-06-02 19:25:22.627978038 +0000 UTC m=+252.717846248" lastFinishedPulling="2026-06-02 19:25:24.365279311 +0000 UTC m=+254.455147521" observedRunningTime="2026-06-02 19:25:25.351164206 +0000 UTC m=+255.441032439" watchObservedRunningTime="2026-06-02 19:25:25.352169487 +0000 UTC m=+255.442037720" Jun 02 19:25:32.957982 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:32.957899 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:25:33.357840 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:33.357792 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="alertmanager" containerID="cri-o://e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1" gracePeriod=120 Jun 02 19:25:33.358060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:33.357867 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy-metric" containerID="cri-o://bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4" gracePeriod=120 Jun 02 19:25:33.358060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:33.357904 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="config-reloader" containerID="cri-o://e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746" gracePeriod=120 Jun 02 19:25:33.358060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:33.357867 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy-web" containerID="cri-o://4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71" gracePeriod=120 Jun 02 19:25:33.358060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:33.357928 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy" containerID="cri-o://b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6" gracePeriod=120 Jun 02 19:25:33.358060 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:33.357931 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="prom-label-proxy" containerID="cri-o://adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c" gracePeriod=120 Jun 02 19:25:34.363550 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.363517 2552 generic.go:358] "Generic (PLEG): container finished" podID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerID="adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c" exitCode=0 Jun 02 19:25:34.363550 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.363532 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerDied","Data":"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c"} Jun 02 19:25:34.363960 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.363574 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerDied","Data":"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6"} Jun 02 19:25:34.363960 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.363543 2552 generic.go:358] "Generic (PLEG): container finished" podID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerID="b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6" exitCode=0 Jun 02 19:25:34.363960 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.363592 2552 generic.go:358] "Generic (PLEG): container finished" podID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerID="e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746" exitCode=0 Jun 02 19:25:34.363960 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.363600 2552 generic.go:358] "Generic (PLEG): container finished" podID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerID="e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1" exitCode=0 Jun 02 19:25:34.363960 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.363676 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerDied","Data":"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746"} Jun 02 19:25:34.363960 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.363685 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerDied","Data":"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1"} Jun 02 19:25:34.612083 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.612058 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:34.738685 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738580 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-main-db\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.738685 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738642 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-tls-assets\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.738685 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738675 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-out\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.738992 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738709 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-web-config\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.738992 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738739 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-volume\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.738992 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738849 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-cluster-tls-config\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.738992 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738926 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-web\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.738992 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738965 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9dkt\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-kube-api-access-t9dkt\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.739270 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.738992 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.739270 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.739008 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:25:34.739270 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.739060 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-trusted-ca-bundle\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.739270 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.739111 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.739270 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.739151 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-metrics-client-ca\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.739270 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.739191 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy\") pod \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\" (UID: \"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab\") " Jun 02 19:25:34.739590 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.739518 2552 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-main-db\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.740228 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.739918 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:25:34.741858 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.741829 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-out" (OuterVolumeSpecName: "config-out") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:25:34.741994 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.741850 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:34.742158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.742131 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-volume" (OuterVolumeSpecName: "config-volume") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:34.742286 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.742261 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:25:34.742539 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.742511 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:34.742539 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.742514 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:25:34.742876 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.742849 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-kube-api-access-t9dkt" (OuterVolumeSpecName: "kube-api-access-t9dkt") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "kube-api-access-t9dkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:25:34.742955 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.742856 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:34.744138 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.744112 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:34.747699 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.747674 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:34.754622 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.754596 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-web-config" (OuterVolumeSpecName: "web-config") pod "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" (UID: "6e63e85f-3e0f-4c91-98c2-36c7a125e7ab"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:25:34.840362 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840321 2552 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840362 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840355 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-main-tls\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840362 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840366 2552 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-metrics-client-ca\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840377 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840388 2552 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-tls-assets\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840397 2552 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-out\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840406 2552 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-web-config\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840416 2552 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-config-volume\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840424 2552 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-cluster-tls-config\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840433 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840442 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9dkt\" (UniqueName: \"kubernetes.io/projected/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-kube-api-access-t9dkt\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:34.840678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:34.840452 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:25:35.370188 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.370151 2552 generic.go:358] "Generic (PLEG): container finished" podID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerID="bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4" exitCode=0 Jun 02 19:25:35.370188 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.370178 2552 generic.go:358] "Generic (PLEG): container finished" podID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerID="4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71" exitCode=0 Jun 02 19:25:35.370735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.370238 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerDied","Data":"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4"} Jun 02 19:25:35.370735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.370266 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.370735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.370287 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerDied","Data":"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71"} Jun 02 19:25:35.370735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.370304 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e63e85f-3e0f-4c91-98c2-36c7a125e7ab","Type":"ContainerDied","Data":"65be35c1516fef08646fcecae0eba9b2726cc800069f8179e73f05c70d73887e"} Jun 02 19:25:35.370735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.370325 2552 scope.go:117] "RemoveContainer" containerID="adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c" Jun 02 19:25:35.379247 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.379115 2552 scope.go:117] "RemoveContainer" containerID="bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4" Jun 02 19:25:35.386811 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.386791 2552 scope.go:117] "RemoveContainer" containerID="b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6" Jun 02 19:25:35.395033 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.395003 2552 scope.go:117] "RemoveContainer" containerID="4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71" Jun 02 19:25:35.398154 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.398124 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:25:35.410867 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.410830 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:25:35.411353 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.411334 2552 scope.go:117] "RemoveContainer" containerID="e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746" Jun 02 19:25:35.419003 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.418982 2552 scope.go:117] "RemoveContainer" containerID="e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1" Jun 02 19:25:35.426737 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.426712 2552 scope.go:117] "RemoveContainer" containerID="56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6" Jun 02 19:25:35.434573 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.434553 2552 scope.go:117] "RemoveContainer" containerID="adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c" Jun 02 19:25:35.434898 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:25:35.434879 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c\": container with ID starting with adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c not found: ID does not exist" containerID="adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c" Jun 02 19:25:35.434942 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.434909 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c"} err="failed to get container status \"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c\": rpc error: code = NotFound desc = could not find container \"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c\": container with ID starting with adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c not found: ID does not exist" Jun 02 19:25:35.434942 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.434931 2552 scope.go:117] "RemoveContainer" containerID="bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4" Jun 02 19:25:35.435194 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:25:35.435174 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4\": container with ID starting with bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4 not found: ID does not exist" containerID="bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4" Jun 02 19:25:35.435235 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.435202 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4"} err="failed to get container status \"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4\": rpc error: code = NotFound desc = could not find container \"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4\": container with ID starting with bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4 not found: ID does not exist" Jun 02 19:25:35.435235 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.435224 2552 scope.go:117] "RemoveContainer" containerID="b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6" Jun 02 19:25:35.435420 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:25:35.435406 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6\": container with ID starting with b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6 not found: ID does not exist" containerID="b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6" Jun 02 19:25:35.435458 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.435424 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6"} err="failed to get container status \"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6\": rpc error: code = NotFound desc = could not find container \"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6\": container with ID starting with b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6 not found: ID does not exist" Jun 02 19:25:35.435458 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.435440 2552 scope.go:117] "RemoveContainer" containerID="4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71" Jun 02 19:25:35.435705 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:25:35.435685 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71\": container with ID starting with 4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71 not found: ID does not exist" containerID="4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71" Jun 02 19:25:35.435793 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.435714 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71"} err="failed to get container status \"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71\": rpc error: code = NotFound desc = could not find container \"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71\": container with ID starting with 4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71 not found: ID does not exist" Jun 02 19:25:35.435793 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.435736 2552 scope.go:117] "RemoveContainer" containerID="e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746" Jun 02 19:25:35.435966 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:25:35.435949 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746\": container with ID starting with e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746 not found: ID does not exist" containerID="e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746" Jun 02 19:25:35.436019 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.435969 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746"} err="failed to get container status \"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746\": rpc error: code = NotFound desc = could not find container \"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746\": container with ID starting with e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746 not found: ID does not exist" Jun 02 19:25:35.436019 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.435984 2552 scope.go:117] "RemoveContainer" containerID="e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1" Jun 02 19:25:35.436220 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:25:35.436202 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1\": container with ID starting with e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1 not found: ID does not exist" containerID="e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1" Jun 02 19:25:35.436260 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.436223 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1"} err="failed to get container status \"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1\": rpc error: code = NotFound desc = could not find container \"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1\": container with ID starting with e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1 not found: ID does not exist" Jun 02 19:25:35.436260 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.436235 2552 scope.go:117] "RemoveContainer" containerID="56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6" Jun 02 19:25:35.436452 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:25:35.436432 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6\": container with ID starting with 56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6 not found: ID does not exist" containerID="56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6" Jun 02 19:25:35.436514 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.436453 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6"} err="failed to get container status \"56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6\": rpc error: code = NotFound desc = could not find container \"56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6\": container with ID starting with 56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6 not found: ID does not exist" Jun 02 19:25:35.436514 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.436464 2552 scope.go:117] "RemoveContainer" containerID="adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c" Jun 02 19:25:35.436687 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.436672 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c"} err="failed to get container status \"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c\": rpc error: code = NotFound desc = could not find container \"adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c\": container with ID starting with adc3cc692c469e4c7f13fcf15e9e1b02ce90d122ab7f980647af2eb84180616c not found: ID does not exist" Jun 02 19:25:35.436760 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.436687 2552 scope.go:117] "RemoveContainer" containerID="bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4" Jun 02 19:25:35.436898 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.436878 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4"} err="failed to get container status \"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4\": rpc error: code = NotFound desc = could not find container \"bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4\": container with ID starting with bcd07e7176a38c278c1228f1d9c539089d84b53ff400614c1e13b33fd8c376c4 not found: ID does not exist" Jun 02 19:25:35.436942 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.436900 2552 scope.go:117] "RemoveContainer" containerID="b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6" Jun 02 19:25:35.437085 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437069 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6"} err="failed to get container status \"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6\": rpc error: code = NotFound desc = could not find container \"b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6\": container with ID starting with b9fdff56658312ac5709f040ae42fa1928cc5638927419e0ee378183b7ecc8c6 not found: ID does not exist" Jun 02 19:25:35.437127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437085 2552 scope.go:117] "RemoveContainer" containerID="4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71" Jun 02 19:25:35.437276 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437260 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71"} err="failed to get container status \"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71\": rpc error: code = NotFound desc = could not find container \"4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71\": container with ID starting with 4f07ca41079305b8e6b10419ee7e515284c1088b1bec015949a52e99510edf71 not found: ID does not exist" Jun 02 19:25:35.437334 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437276 2552 scope.go:117] "RemoveContainer" containerID="e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746" Jun 02 19:25:35.437459 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437441 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746"} err="failed to get container status \"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746\": rpc error: code = NotFound desc = could not find container \"e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746\": container with ID starting with e1c0106fe438fac17f1931f9d245d04902c78ad88a17f6335a7309539eeac746 not found: ID does not exist" Jun 02 19:25:35.437548 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437460 2552 scope.go:117] "RemoveContainer" containerID="e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1" Jun 02 19:25:35.437683 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437666 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1"} err="failed to get container status \"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1\": rpc error: code = NotFound desc = could not find container \"e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1\": container with ID starting with e6006249b2649c8872cceefcdd3b67e2f381dcb9f3f482be55d902ffb6c19da1 not found: ID does not exist" Jun 02 19:25:35.437721 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437683 2552 scope.go:117] "RemoveContainer" containerID="56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6" Jun 02 19:25:35.437875 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.437857 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6"} err="failed to get container status \"56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6\": rpc error: code = NotFound desc = could not find container \"56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6\": container with ID starting with 56b43b9a7081de665f987c724339674c5813e92288205b7bf22d52e83e447ba6 not found: ID does not exist" Jun 02 19:25:35.443316 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443287 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:25:35.443644 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443630 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="config-reloader" Jun 02 19:25:35.443704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443646 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="config-reloader" Jun 02 19:25:35.443704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443658 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy-metric" Jun 02 19:25:35.443704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443664 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy-metric" Jun 02 19:25:35.443704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443674 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="prom-label-proxy" Jun 02 19:25:35.443704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443680 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="prom-label-proxy" Jun 02 19:25:35.443704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443689 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" containerName="console" Jun 02 19:25:35.443704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443695 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" containerName="console" Jun 02 19:25:35.443704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443702 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443708 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443716 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="alertmanager" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443721 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="alertmanager" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443728 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="init-config-reloader" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443734 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="init-config-reloader" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443749 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy-web" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443753 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy-web" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443806 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy-web" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443814 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="alertmanager" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443822 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443829 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="config-reloader" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443835 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="prom-label-proxy" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443841 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" containerName="kube-rbac-proxy-metric" Jun 02 19:25:35.443963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.443847 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="2e36f3d4-d8bf-4cd6-8575-bcb6f836959f" containerName="console" Jun 02 19:25:35.449272 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.449255 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.453547 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.453525 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 19:25:35.453667 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.453593 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 19:25:35.453825 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.453809 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 19:25:35.454252 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.454227 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 19:25:35.454363 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.454227 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 19:25:35.454363 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.454286 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 19:25:35.454594 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.454574 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 19:25:35.454594 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.454586 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 19:25:35.457518 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.457475 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-v558n\"" Jun 02 19:25:35.461692 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.461673 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 19:25:35.466974 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.466944 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:25:35.546762 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.546719 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.546762 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.546766 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.546974 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.546812 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbj2n\" (UniqueName: \"kubernetes.io/projected/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-kube-api-access-xbj2n\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.546974 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.546868 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.546974 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.546929 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.546974 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.546964 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-config-out\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.547094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.546989 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.547094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.547008 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-web-config\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.547094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.547052 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.547094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.547085 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.547217 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.547101 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.547217 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.547119 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.547217 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.547150 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648020 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.647925 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-config-out\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648020 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.647972 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648020 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648004 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-web-config\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648218 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648051 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648218 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648095 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648218 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648117 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648218 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648144 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648218 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648173 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648218 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648211 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648487 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648246 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648487 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648294 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xbj2n\" (UniqueName: \"kubernetes.io/projected/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-kube-api-access-xbj2n\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648487 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648331 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648487 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648356 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.648752 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.648537 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.649100 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.649071 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.649846 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.649774 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.651265 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.651237 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-web-config\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.651383 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.651357 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.651600 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.651577 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-config-out\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.651694 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.651632 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.651842 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.651819 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.651998 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.651980 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.652205 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.652185 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.652248 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.652233 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.653385 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.653368 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.659048 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.659025 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbj2n\" (UniqueName: \"kubernetes.io/projected/9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3-kube-api-access-xbj2n\") pod \"alertmanager-main-0\" (UID: \"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.759389 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.759357 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:25:35.932706 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:35.932681 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:25:35.935066 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:25:35.935039 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6fe7a8_541e_4225_bc33_a8aba2e8c6f3.slice/crio-afd948be9e8414f33a71042bd4bea549f74520581de086545cd43c74155186a7 WatchSource:0}: Error finding container afd948be9e8414f33a71042bd4bea549f74520581de086545cd43c74155186a7: Status 404 returned error can't find the container with id afd948be9e8414f33a71042bd4bea549f74520581de086545cd43c74155186a7 Jun 02 19:25:36.374287 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.374251 2552 generic.go:358] "Generic (PLEG): container finished" podID="9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3" containerID="8cc4644d0a534ae57d4b9f797e51db730e6fdf63af96fdad1b8d0cfe7f97b5f6" exitCode=0 Jun 02 19:25:36.374727 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.374329 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3","Type":"ContainerDied","Data":"8cc4644d0a534ae57d4b9f797e51db730e6fdf63af96fdad1b8d0cfe7f97b5f6"} Jun 02 19:25:36.374727 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.374359 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3","Type":"ContainerStarted","Data":"afd948be9e8414f33a71042bd4bea549f74520581de086545cd43c74155186a7"} Jun 02 19:25:36.396042 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.396009 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e63e85f-3e0f-4c91-98c2-36c7a125e7ab" path="/var/lib/kubelet/pods/6e63e85f-3e0f-4c91-98c2-36c7a125e7ab/volumes" Jun 02 19:25:36.824689 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.824646 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-795c9c8864-9trx6"] Jun 02 19:25:36.828299 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.827927 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:36.837048 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.837020 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-795c9c8864-9trx6"] Jun 02 19:25:36.964032 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.963955 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-service-ca\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:36.964032 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.963998 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-serving-cert\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:36.964204 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.964124 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-oauth-config\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:36.964204 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.964154 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-oauth-serving-cert\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:36.964204 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.964171 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxkgq\" (UniqueName: \"kubernetes.io/projected/19e20106-ec71-4202-aabd-914e728ac6de-kube-api-access-fxkgq\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:36.964350 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.964232 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-console-config\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:36.964350 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:36.964260 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-trusted-ca-bundle\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.064839 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.064810 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-oauth-config\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.064839 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.064844 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-oauth-serving-cert\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.065058 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.064860 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fxkgq\" (UniqueName: \"kubernetes.io/projected/19e20106-ec71-4202-aabd-914e728ac6de-kube-api-access-fxkgq\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.065058 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.064883 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-console-config\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.065058 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.064914 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-trusted-ca-bundle\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.065058 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.064950 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-service-ca\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.065058 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.064976 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-serving-cert\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.065705 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.065677 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-service-ca\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.065834 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.065782 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-console-config\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.065897 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.065864 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-oauth-serving-cert\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.066096 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.066071 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-trusted-ca-bundle\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.067349 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.067325 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-oauth-config\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.067481 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.067464 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-serving-cert\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.072991 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.072974 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxkgq\" (UniqueName: \"kubernetes.io/projected/19e20106-ec71-4202-aabd-914e728ac6de-kube-api-access-fxkgq\") pod \"console-795c9c8864-9trx6\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.138689 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.138658 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:37.268183 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.268158 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-795c9c8864-9trx6"] Jun 02 19:25:37.270781 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:25:37.270756 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e20106_ec71_4202_aabd_914e728ac6de.slice/crio-d5793d83979feec3c764c7558572b47f1c5ca6490fe2716b8753769d0a063ddb WatchSource:0}: Error finding container d5793d83979feec3c764c7558572b47f1c5ca6490fe2716b8753769d0a063ddb: Status 404 returned error can't find the container with id d5793d83979feec3c764c7558572b47f1c5ca6490fe2716b8753769d0a063ddb Jun 02 19:25:37.382637 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.382595 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3","Type":"ContainerStarted","Data":"6a9597f6721171c3b4b323bd6ac06871e2363e8cc8112425e55bb700e3ce196f"} Jun 02 19:25:37.382637 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.382638 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3","Type":"ContainerStarted","Data":"947e6d99e2f0b2ba9c9ba67ba474f5f13484accfcebf56dc5040fe129c9fa247"} Jun 02 19:25:37.383130 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.382652 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3","Type":"ContainerStarted","Data":"4284477812ac843b4017bcda831aa33b22c3e7323514bb16bf142ebae1039c24"} Jun 02 19:25:37.383130 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.382669 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3","Type":"ContainerStarted","Data":"05a6592717a34801442bd4e989f32048876784e6bc44b1926e9e4a70e26bc392"} Jun 02 19:25:37.383130 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.382683 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3","Type":"ContainerStarted","Data":"c8ed459aab5a761d89c7e07085f74b91a6cba1c8edf6cbc1b96c086b49f51751"} Jun 02 19:25:37.383130 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.382695 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3","Type":"ContainerStarted","Data":"5c0388b254c2a8b17ba19264a22298db9e2f791e3eff80a588b01d2045439283"} Jun 02 19:25:37.383944 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.383912 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795c9c8864-9trx6" event={"ID":"19e20106-ec71-4202-aabd-914e728ac6de","Type":"ContainerStarted","Data":"7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1"} Jun 02 19:25:37.384064 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.383949 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795c9c8864-9trx6" event={"ID":"19e20106-ec71-4202-aabd-914e728ac6de","Type":"ContainerStarted","Data":"d5793d83979feec3c764c7558572b47f1c5ca6490fe2716b8753769d0a063ddb"} Jun 02 19:25:37.416292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.416246 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.416231805 podStartE2EDuration="2.416231805s" podCreationTimestamp="2026-06-02 19:25:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:25:37.415033282 +0000 UTC m=+267.504901515" watchObservedRunningTime="2026-06-02 19:25:37.416231805 +0000 UTC m=+267.506100037" Jun 02 19:25:37.439114 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:37.439049 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-795c9c8864-9trx6" podStartSLOduration=1.439030905 podStartE2EDuration="1.439030905s" podCreationTimestamp="2026-06-02 19:25:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:25:37.437176348 +0000 UTC m=+267.527044584" watchObservedRunningTime="2026-06-02 19:25:37.439030905 +0000 UTC m=+267.528899139" Jun 02 19:25:47.138875 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:47.138833 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:47.139480 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:47.138889 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:47.143867 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:47.143841 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:47.423973 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:47.423879 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:25:47.475681 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:25:47.475642 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d78699465-mm9ch"] Jun 02 19:26:10.277191 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:10.277164 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:26:10.281869 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:10.281845 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:26:10.282975 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:10.282951 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:26:10.287861 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:10.287839 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:26:10.287970 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:10.287912 2552 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 19:26:12.501179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.501115 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-d78699465-mm9ch" podUID="5d835815-c2ab-4c91-bd53-e991b93314c9" containerName="console" containerID="cri-o://150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4" gracePeriod=15 Jun 02 19:26:12.740583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.740563 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d78699465-mm9ch_5d835815-c2ab-4c91-bd53-e991b93314c9/console/0.log" Jun 02 19:26:12.740705 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.740626 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:26:12.781598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.781520 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-trusted-ca-bundle\") pod \"5d835815-c2ab-4c91-bd53-e991b93314c9\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " Jun 02 19:26:12.781598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.781579 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-oauth-serving-cert\") pod \"5d835815-c2ab-4c91-bd53-e991b93314c9\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " Jun 02 19:26:12.781809 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.781616 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-serving-cert\") pod \"5d835815-c2ab-4c91-bd53-e991b93314c9\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " Jun 02 19:26:12.781809 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.781640 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8hk2\" (UniqueName: \"kubernetes.io/projected/5d835815-c2ab-4c91-bd53-e991b93314c9-kube-api-access-r8hk2\") pod \"5d835815-c2ab-4c91-bd53-e991b93314c9\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " Jun 02 19:26:12.781809 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.781670 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-oauth-config\") pod \"5d835815-c2ab-4c91-bd53-e991b93314c9\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " Jun 02 19:26:12.781809 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.781747 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-console-config\") pod \"5d835815-c2ab-4c91-bd53-e991b93314c9\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " Jun 02 19:26:12.781809 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.781791 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-service-ca\") pod \"5d835815-c2ab-4c91-bd53-e991b93314c9\" (UID: \"5d835815-c2ab-4c91-bd53-e991b93314c9\") " Jun 02 19:26:12.782041 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.781998 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5d835815-c2ab-4c91-bd53-e991b93314c9" (UID: "5d835815-c2ab-4c91-bd53-e991b93314c9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:26:12.782192 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.782155 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5d835815-c2ab-4c91-bd53-e991b93314c9" (UID: "5d835815-c2ab-4c91-bd53-e991b93314c9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:26:12.782325 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.782260 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-console-config" (OuterVolumeSpecName: "console-config") pod "5d835815-c2ab-4c91-bd53-e991b93314c9" (UID: "5d835815-c2ab-4c91-bd53-e991b93314c9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:26:12.782377 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.782319 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-service-ca" (OuterVolumeSpecName: "service-ca") pod "5d835815-c2ab-4c91-bd53-e991b93314c9" (UID: "5d835815-c2ab-4c91-bd53-e991b93314c9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:26:12.783912 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.783884 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5d835815-c2ab-4c91-bd53-e991b93314c9" (UID: "5d835815-c2ab-4c91-bd53-e991b93314c9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:26:12.784001 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.783947 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5d835815-c2ab-4c91-bd53-e991b93314c9" (UID: "5d835815-c2ab-4c91-bd53-e991b93314c9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:26:12.784174 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.784156 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d835815-c2ab-4c91-bd53-e991b93314c9-kube-api-access-r8hk2" (OuterVolumeSpecName: "kube-api-access-r8hk2") pod "5d835815-c2ab-4c91-bd53-e991b93314c9" (UID: "5d835815-c2ab-4c91-bd53-e991b93314c9"). InnerVolumeSpecName "kube-api-access-r8hk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:26:12.882574 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.882533 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-console-config\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:26:12.882574 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.882564 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-service-ca\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:26:12.882574 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.882573 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-trusted-ca-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:26:12.882574 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.882584 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d835815-c2ab-4c91-bd53-e991b93314c9-oauth-serving-cert\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:26:12.882832 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.882593 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-serving-cert\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:26:12.882832 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.882602 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8hk2\" (UniqueName: \"kubernetes.io/projected/5d835815-c2ab-4c91-bd53-e991b93314c9-kube-api-access-r8hk2\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:26:12.882832 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:12.882610 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d835815-c2ab-4c91-bd53-e991b93314c9-console-oauth-config\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:26:13.509192 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.509158 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d78699465-mm9ch_5d835815-c2ab-4c91-bd53-e991b93314c9/console/0.log" Jun 02 19:26:13.509649 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.509202 2552 generic.go:358] "Generic (PLEG): container finished" podID="5d835815-c2ab-4c91-bd53-e991b93314c9" containerID="150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4" exitCode=2 Jun 02 19:26:13.509649 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.509291 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d78699465-mm9ch" Jun 02 19:26:13.509649 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.509288 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d78699465-mm9ch" event={"ID":"5d835815-c2ab-4c91-bd53-e991b93314c9","Type":"ContainerDied","Data":"150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4"} Jun 02 19:26:13.509649 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.509389 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d78699465-mm9ch" event={"ID":"5d835815-c2ab-4c91-bd53-e991b93314c9","Type":"ContainerDied","Data":"76ae8aa83a9da05455ddddfe2df9397c750d1454b9e2682653f1a718edcc01ab"} Jun 02 19:26:13.509649 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.509404 2552 scope.go:117] "RemoveContainer" containerID="150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4" Jun 02 19:26:13.520635 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.520618 2552 scope.go:117] "RemoveContainer" containerID="150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4" Jun 02 19:26:13.520916 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:26:13.520898 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4\": container with ID starting with 150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4 not found: ID does not exist" containerID="150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4" Jun 02 19:26:13.520964 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.520925 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4"} err="failed to get container status \"150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4\": rpc error: code = NotFound desc = could not find container \"150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4\": container with ID starting with 150de257efe3eaa621460243c1373070431e0cda8a62e8c7f5eefc88a04967a4 not found: ID does not exist" Jun 02 19:26:13.534160 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.534138 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d78699465-mm9ch"] Jun 02 19:26:13.537914 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:13.537894 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d78699465-mm9ch"] Jun 02 19:26:14.394316 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:26:14.394281 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d835815-c2ab-4c91-bd53-e991b93314c9" path="/var/lib/kubelet/pods/5d835815-c2ab-4c91-bd53-e991b93314c9/volumes" Jun 02 19:27:19.767129 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.767090 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9"] Jun 02 19:27:19.767669 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.767451 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d835815-c2ab-4c91-bd53-e991b93314c9" containerName="console" Jun 02 19:27:19.767669 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.767462 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d835815-c2ab-4c91-bd53-e991b93314c9" containerName="console" Jun 02 19:27:19.767669 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.767526 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="5d835815-c2ab-4c91-bd53-e991b93314c9" containerName="console" Jun 02 19:27:19.770819 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.770795 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" Jun 02 19:27:19.775702 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.775679 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 02 19:27:19.775820 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.775770 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lzwjr\"" Jun 02 19:27:19.776040 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.776021 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:27:19.787852 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.787823 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9"] Jun 02 19:27:19.883179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.883138 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdca22e-2b9c-442c-935c-9071246bc816-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-9rwg9\" (UID: \"bbdca22e-2b9c-442c-935c-9071246bc816\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" Jun 02 19:27:19.883374 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.883223 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w94d\" (UniqueName: \"kubernetes.io/projected/bbdca22e-2b9c-442c-935c-9071246bc816-kube-api-access-5w94d\") pod \"cert-manager-operator-controller-manager-54b9655956-9rwg9\" (UID: \"bbdca22e-2b9c-442c-935c-9071246bc816\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" Jun 02 19:27:19.984187 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.984150 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5w94d\" (UniqueName: \"kubernetes.io/projected/bbdca22e-2b9c-442c-935c-9071246bc816-kube-api-access-5w94d\") pod \"cert-manager-operator-controller-manager-54b9655956-9rwg9\" (UID: \"bbdca22e-2b9c-442c-935c-9071246bc816\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" Jun 02 19:27:19.984364 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.984218 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdca22e-2b9c-442c-935c-9071246bc816-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-9rwg9\" (UID: \"bbdca22e-2b9c-442c-935c-9071246bc816\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" Jun 02 19:27:19.984598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.984582 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdca22e-2b9c-442c-935c-9071246bc816-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-9rwg9\" (UID: \"bbdca22e-2b9c-442c-935c-9071246bc816\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" Jun 02 19:27:19.992702 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:19.992676 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w94d\" (UniqueName: \"kubernetes.io/projected/bbdca22e-2b9c-442c-935c-9071246bc816-kube-api-access-5w94d\") pod \"cert-manager-operator-controller-manager-54b9655956-9rwg9\" (UID: \"bbdca22e-2b9c-442c-935c-9071246bc816\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" Jun 02 19:27:20.080558 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:20.080521 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" Jun 02 19:27:20.216239 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:20.216211 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9"] Jun 02 19:27:20.218747 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:27:20.218716 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdca22e_2b9c_442c_935c_9071246bc816.slice/crio-b4eebb6c44cdeaac1d0bb7839937d2d29b25c7bc7d656dc17b6d0f2db6e2404d WatchSource:0}: Error finding container b4eebb6c44cdeaac1d0bb7839937d2d29b25c7bc7d656dc17b6d0f2db6e2404d: Status 404 returned error can't find the container with id b4eebb6c44cdeaac1d0bb7839937d2d29b25c7bc7d656dc17b6d0f2db6e2404d Jun 02 19:27:20.222300 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:20.222284 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:27:20.716587 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:20.716552 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" event={"ID":"bbdca22e-2b9c-442c-935c-9071246bc816","Type":"ContainerStarted","Data":"b4eebb6c44cdeaac1d0bb7839937d2d29b25c7bc7d656dc17b6d0f2db6e2404d"} Jun 02 19:27:23.728296 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:23.728251 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" event={"ID":"bbdca22e-2b9c-442c-935c-9071246bc816","Type":"ContainerStarted","Data":"5698f6c16bb3c06a3731b1a0014aa06b5ca96fb09b71ad9876f2e759591b07a6"} Jun 02 19:27:23.752179 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:23.752122 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-9rwg9" podStartSLOduration=2.196622721 podStartE2EDuration="4.752102708s" podCreationTimestamp="2026-06-02 19:27:19 +0000 UTC" firstStartedPulling="2026-06-02 19:27:20.222452448 +0000 UTC m=+370.312320658" lastFinishedPulling="2026-06-02 19:27:22.777932431 +0000 UTC m=+372.867800645" observedRunningTime="2026-06-02 19:27:23.750669275 +0000 UTC m=+373.840537509" watchObservedRunningTime="2026-06-02 19:27:23.752102708 +0000 UTC m=+373.841970940" Jun 02 19:27:24.305899 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.305862 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm"] Jun 02 19:27:24.309626 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.309605 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.313692 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.313659 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 19:27:24.314488 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.314471 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 19:27:24.314584 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.314473 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qd585\"" Jun 02 19:27:24.320245 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.320219 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm"] Jun 02 19:27:24.429105 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.429072 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.429286 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.429131 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5h2w\" (UniqueName: \"kubernetes.io/projected/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-kube-api-access-s5h2w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.429286 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.429220 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.530308 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.530268 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.530550 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.530335 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s5h2w\" (UniqueName: \"kubernetes.io/projected/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-kube-api-access-s5h2w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.530550 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.530397 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.530701 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.530680 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.530759 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.530744 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.545354 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.545321 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5h2w\" (UniqueName: \"kubernetes.io/projected/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-kube-api-access-s5h2w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.619096 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.619051 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:27:24.755626 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:24.755598 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm"] Jun 02 19:27:24.758100 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:27:24.758060 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda366e26d_5d0f_4d2e_b919_c5be3e6d4090.slice/crio-f23942c13837d660001f20702888c26664bf182bad6758a6766c09dc9e22f87d WatchSource:0}: Error finding container f23942c13837d660001f20702888c26664bf182bad6758a6766c09dc9e22f87d: Status 404 returned error can't find the container with id f23942c13837d660001f20702888c26664bf182bad6758a6766c09dc9e22f87d Jun 02 19:27:25.737896 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:25.737836 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" event={"ID":"a366e26d-5d0f-4d2e-b919-c5be3e6d4090","Type":"ContainerStarted","Data":"f23942c13837d660001f20702888c26664bf182bad6758a6766c09dc9e22f87d"} Jun 02 19:27:28.863052 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.863008 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-q4nb9"] Jun 02 19:27:28.866654 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.866621 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" Jun 02 19:27:28.869399 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.869373 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 02 19:27:28.869399 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.869393 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 02 19:27:28.869603 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.869430 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfw5q\" (UniqueName: \"kubernetes.io/projected/2c60682f-8cf2-4039-a879-1824749d6cf2-kube-api-access-vfw5q\") pod \"cert-manager-cainjector-68b757865b-q4nb9\" (UID: \"2c60682f-8cf2-4039-a879-1824749d6cf2\") " pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" Jun 02 19:27:28.869603 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.869542 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c60682f-8cf2-4039-a879-1824749d6cf2-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-q4nb9\" (UID: \"2c60682f-8cf2-4039-a879-1824749d6cf2\") " pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" Jun 02 19:27:28.870435 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.870419 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4rd8d\"" Jun 02 19:27:28.876421 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.876140 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-q4nb9"] Jun 02 19:27:28.970181 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.970145 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c60682f-8cf2-4039-a879-1824749d6cf2-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-q4nb9\" (UID: \"2c60682f-8cf2-4039-a879-1824749d6cf2\") " pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" Jun 02 19:27:28.970396 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.970241 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfw5q\" (UniqueName: \"kubernetes.io/projected/2c60682f-8cf2-4039-a879-1824749d6cf2-kube-api-access-vfw5q\") pod \"cert-manager-cainjector-68b757865b-q4nb9\" (UID: \"2c60682f-8cf2-4039-a879-1824749d6cf2\") " pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" Jun 02 19:27:28.978117 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.978087 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c60682f-8cf2-4039-a879-1824749d6cf2-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-q4nb9\" (UID: \"2c60682f-8cf2-4039-a879-1824749d6cf2\") " pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" Jun 02 19:27:28.978239 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:28.978220 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfw5q\" (UniqueName: \"kubernetes.io/projected/2c60682f-8cf2-4039-a879-1824749d6cf2-kube-api-access-vfw5q\") pod \"cert-manager-cainjector-68b757865b-q4nb9\" (UID: \"2c60682f-8cf2-4039-a879-1824749d6cf2\") " pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" Jun 02 19:27:29.188966 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:29.188878 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" Jun 02 19:27:29.331841 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:29.331802 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-q4nb9"] Jun 02 19:27:31.124482 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:27:31.124440 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c60682f_8cf2_4039_a879_1824749d6cf2.slice/crio-da4b0fcfc8a1de697e2e9865bfda5ba75782191031321e82ad34338f4c0b2a6f WatchSource:0}: Error finding container da4b0fcfc8a1de697e2e9865bfda5ba75782191031321e82ad34338f4c0b2a6f: Status 404 returned error can't find the container with id da4b0fcfc8a1de697e2e9865bfda5ba75782191031321e82ad34338f4c0b2a6f Jun 02 19:27:31.760173 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:31.760134 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" event={"ID":"2c60682f-8cf2-4039-a879-1824749d6cf2","Type":"ContainerStarted","Data":"da4b0fcfc8a1de697e2e9865bfda5ba75782191031321e82ad34338f4c0b2a6f"} Jun 02 19:27:31.761544 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:31.761510 2552 generic.go:358] "Generic (PLEG): container finished" podID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerID="ca3a6cd7533e42941a9a26be1dbee8c8968b8cf66c9f2539d426bdd72ca4c86e" exitCode=0 Jun 02 19:27:31.761680 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:31.761579 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" event={"ID":"a366e26d-5d0f-4d2e-b919-c5be3e6d4090","Type":"ContainerDied","Data":"ca3a6cd7533e42941a9a26be1dbee8c8968b8cf66c9f2539d426bdd72ca4c86e"} Jun 02 19:27:34.774793 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:34.774755 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" event={"ID":"2c60682f-8cf2-4039-a879-1824749d6cf2","Type":"ContainerStarted","Data":"2a7d0392596ffd44f9f78b34b26fb1f1e99e85d5e03658cd2d55e602baa2194b"} Jun 02 19:27:34.776436 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:34.776405 2552 generic.go:358] "Generic (PLEG): container finished" podID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerID="76f835e7ca2e1f412a7c586326d6a8a2cd6aba6e984c7094a704296db8e788e4" exitCode=0 Jun 02 19:27:34.776587 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:34.776484 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" event={"ID":"a366e26d-5d0f-4d2e-b919-c5be3e6d4090","Type":"ContainerDied","Data":"76f835e7ca2e1f412a7c586326d6a8a2cd6aba6e984c7094a704296db8e788e4"} Jun 02 19:27:34.794955 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:27:34.794894 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-q4nb9" podStartSLOduration=3.346639389 podStartE2EDuration="6.794879077s" podCreationTimestamp="2026-06-02 19:27:28 +0000 UTC" firstStartedPulling="2026-06-02 19:27:31.126397004 +0000 UTC m=+381.216265220" lastFinishedPulling="2026-06-02 19:27:34.574636691 +0000 UTC m=+384.664504908" observedRunningTime="2026-06-02 19:27:34.791861257 +0000 UTC m=+384.881729494" watchObservedRunningTime="2026-06-02 19:27:34.794879077 +0000 UTC m=+384.884747310" Jun 02 19:28:18.930690 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:18.930649 2552 generic.go:358] "Generic (PLEG): container finished" podID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerID="fe1e0fd0215b2d8ca571e2c90c4ed3e3688f081673a40f36472c0d6e777dd9a3" exitCode=0 Jun 02 19:28:18.931138 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:18.930715 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" event={"ID":"a366e26d-5d0f-4d2e-b919-c5be3e6d4090","Type":"ContainerDied","Data":"fe1e0fd0215b2d8ca571e2c90c4ed3e3688f081673a40f36472c0d6e777dd9a3"} Jun 02 19:28:20.059637 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.059610 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:28:20.136361 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.136320 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5h2w\" (UniqueName: \"kubernetes.io/projected/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-kube-api-access-s5h2w\") pod \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " Jun 02 19:28:20.136588 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.136382 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-bundle\") pod \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " Jun 02 19:28:20.136588 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.136415 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-util\") pod \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\" (UID: \"a366e26d-5d0f-4d2e-b919-c5be3e6d4090\") " Jun 02 19:28:20.136878 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.136849 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-bundle" (OuterVolumeSpecName: "bundle") pod "a366e26d-5d0f-4d2e-b919-c5be3e6d4090" (UID: "a366e26d-5d0f-4d2e-b919-c5be3e6d4090"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:28:20.138688 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.138661 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-kube-api-access-s5h2w" (OuterVolumeSpecName: "kube-api-access-s5h2w") pod "a366e26d-5d0f-4d2e-b919-c5be3e6d4090" (UID: "a366e26d-5d0f-4d2e-b919-c5be3e6d4090"). InnerVolumeSpecName "kube-api-access-s5h2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:28:20.140585 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.140557 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-util" (OuterVolumeSpecName: "util") pod "a366e26d-5d0f-4d2e-b919-c5be3e6d4090" (UID: "a366e26d-5d0f-4d2e-b919-c5be3e6d4090"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:28:20.237931 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.237813 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:28:20.237931 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.237862 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-util\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:28:20.237931 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.237873 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5h2w\" (UniqueName: \"kubernetes.io/projected/a366e26d-5d0f-4d2e-b919-c5be3e6d4090-kube-api-access-s5h2w\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:28:20.940828 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.940797 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" Jun 02 19:28:20.940828 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.940809 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl6wvm" event={"ID":"a366e26d-5d0f-4d2e-b919-c5be3e6d4090","Type":"ContainerDied","Data":"f23942c13837d660001f20702888c26664bf182bad6758a6766c09dc9e22f87d"} Jun 02 19:28:20.941044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:20.940840 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f23942c13837d660001f20702888c26664bf182bad6758a6766c09dc9e22f87d" Jun 02 19:28:43.875985 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.875900 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk"] Jun 02 19:28:43.876433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.876240 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerName="extract" Jun 02 19:28:43.876433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.876250 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerName="extract" Jun 02 19:28:43.876433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.876260 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerName="util" Jun 02 19:28:43.876433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.876265 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerName="util" Jun 02 19:28:43.876433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.876283 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerName="pull" Jun 02 19:28:43.876433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.876289 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerName="pull" Jun 02 19:28:43.876433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.876353 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="a366e26d-5d0f-4d2e-b919-c5be3e6d4090" containerName="extract" Jun 02 19:28:43.879376 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.879356 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:43.883389 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.883352 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pvrvq\"" Jun 02 19:28:43.883389 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.883374 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 19:28:43.883639 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.883428 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 19:28:43.883639 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.883486 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 19:28:43.883639 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.883624 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 19:28:43.896480 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.896449 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk"] Jun 02 19:28:43.945249 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.945207 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-tmp\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:43.945249 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.945250 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:43.945486 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.945273 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-webhook-cert\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:43.945486 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.945375 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb5c5\" (UniqueName: \"kubernetes.io/projected/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-kube-api-access-mb5c5\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:43.945486 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:43.945457 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-manifests\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.046874 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.046826 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-tmp\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.046874 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.046876 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.047113 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.046896 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-webhook-cert\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.047113 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.046930 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mb5c5\" (UniqueName: \"kubernetes.io/projected/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-kube-api-access-mb5c5\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.047113 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.046979 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-manifests\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.047423 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.047375 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-manifests\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.049204 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.049172 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-tmp\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.049540 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.049518 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-webhook-cert\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.049592 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.049518 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.057691 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.057667 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb5c5\" (UniqueName: \"kubernetes.io/projected/c2fb622a-a5fd-4129-92e5-fe19e9c72ae7-kube-api-access-mb5c5\") pod \"opendatahub-operator-controller-manager-b5889499b-s48jk\" (UID: \"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7\") " pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.190153 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.190052 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:44.329164 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:44.329052 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk"] Jun 02 19:28:44.331996 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:28:44.331964 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2fb622a_a5fd_4129_92e5_fe19e9c72ae7.slice/crio-09e86a5bce85278718bd9d4d63e5f881e7004a8c986d4060622a37f09d129e5f WatchSource:0}: Error finding container 09e86a5bce85278718bd9d4d63e5f881e7004a8c986d4060622a37f09d129e5f: Status 404 returned error can't find the container with id 09e86a5bce85278718bd9d4d63e5f881e7004a8c986d4060622a37f09d129e5f Jun 02 19:28:45.024036 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:45.023977 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" event={"ID":"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7","Type":"ContainerStarted","Data":"09e86a5bce85278718bd9d4d63e5f881e7004a8c986d4060622a37f09d129e5f"} Jun 02 19:28:48.037429 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:48.037387 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" event={"ID":"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7","Type":"ContainerStarted","Data":"9396828c489b64d041d1820a72ec4200c07ad6204a6f8223f1217debed3a30e9"} Jun 02 19:28:49.041995 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:49.041961 2552 generic.go:358] "Generic (PLEG): container finished" podID="c2fb622a-a5fd-4129-92e5-fe19e9c72ae7" containerID="9396828c489b64d041d1820a72ec4200c07ad6204a6f8223f1217debed3a30e9" exitCode=0 Jun 02 19:28:49.042391 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:49.042041 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" event={"ID":"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7","Type":"ContainerDied","Data":"9396828c489b64d041d1820a72ec4200c07ad6204a6f8223f1217debed3a30e9"} Jun 02 19:28:50.047316 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:50.047279 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" event={"ID":"c2fb622a-a5fd-4129-92e5-fe19e9c72ae7","Type":"ContainerStarted","Data":"954b1c9a992ad22424162400847584715196c2e0e5fcdb27f32329c13e2ebf7e"} Jun 02 19:28:50.047736 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:50.047407 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:28:50.076280 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:28:50.076229 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" podStartSLOduration=2.02400551 podStartE2EDuration="7.076208015s" podCreationTimestamp="2026-06-02 19:28:43 +0000 UTC" firstStartedPulling="2026-06-02 19:28:44.334255759 +0000 UTC m=+454.424123970" lastFinishedPulling="2026-06-02 19:28:49.38645826 +0000 UTC m=+459.476326475" observedRunningTime="2026-06-02 19:28:50.072187641 +0000 UTC m=+460.162055874" watchObservedRunningTime="2026-06-02 19:28:50.076208015 +0000 UTC m=+460.166076250" Jun 02 19:29:01.053865 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:01.053829 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b5889499b-s48jk" Jun 02 19:29:03.429196 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.429151 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-bd4dl"] Jun 02 19:29:03.432725 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.432706 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:03.435258 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.435227 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 02 19:29:03.435394 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.435270 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-d599c\"" Jun 02 19:29:03.442865 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.442826 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-bd4dl"] Jun 02 19:29:03.528343 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.528293 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbzf6\" (UniqueName: \"kubernetes.io/projected/37f23b73-2276-47ef-9c7b-0dc0910007f1-kube-api-access-wbzf6\") pod \"odh-model-controller-76588b95-bd4dl\" (UID: \"37f23b73-2276-47ef-9c7b-0dc0910007f1\") " pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:03.528576 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.528383 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37f23b73-2276-47ef-9c7b-0dc0910007f1-cert\") pod \"odh-model-controller-76588b95-bd4dl\" (UID: \"37f23b73-2276-47ef-9c7b-0dc0910007f1\") " pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:03.629953 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.629918 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbzf6\" (UniqueName: \"kubernetes.io/projected/37f23b73-2276-47ef-9c7b-0dc0910007f1-kube-api-access-wbzf6\") pod \"odh-model-controller-76588b95-bd4dl\" (UID: \"37f23b73-2276-47ef-9c7b-0dc0910007f1\") " pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:03.630140 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.629989 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37f23b73-2276-47ef-9c7b-0dc0910007f1-cert\") pod \"odh-model-controller-76588b95-bd4dl\" (UID: \"37f23b73-2276-47ef-9c7b-0dc0910007f1\") " pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:03.630140 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:29:03.630109 2552 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 02 19:29:03.630223 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:29:03.630212 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f23b73-2276-47ef-9c7b-0dc0910007f1-cert podName:37f23b73-2276-47ef-9c7b-0dc0910007f1 nodeName:}" failed. No retries permitted until 2026-06-02 19:29:04.130195403 +0000 UTC m=+474.220063613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/37f23b73-2276-47ef-9c7b-0dc0910007f1-cert") pod "odh-model-controller-76588b95-bd4dl" (UID: "37f23b73-2276-47ef-9c7b-0dc0910007f1") : secret "odh-model-controller-webhook-cert" not found Jun 02 19:29:03.640400 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:03.640374 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbzf6\" (UniqueName: \"kubernetes.io/projected/37f23b73-2276-47ef-9c7b-0dc0910007f1-kube-api-access-wbzf6\") pod \"odh-model-controller-76588b95-bd4dl\" (UID: \"37f23b73-2276-47ef-9c7b-0dc0910007f1\") " pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:04.134115 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.134073 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37f23b73-2276-47ef-9c7b-0dc0910007f1-cert\") pod \"odh-model-controller-76588b95-bd4dl\" (UID: \"37f23b73-2276-47ef-9c7b-0dc0910007f1\") " pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:04.136678 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.136646 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37f23b73-2276-47ef-9c7b-0dc0910007f1-cert\") pod \"odh-model-controller-76588b95-bd4dl\" (UID: \"37f23b73-2276-47ef-9c7b-0dc0910007f1\") " pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:04.244836 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.244790 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-6577596cd5-8wwnl"] Jun 02 19:29:04.255938 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.255894 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:04.257235 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.257204 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-8wwnl"] Jun 02 19:29:04.259965 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.259931 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 02 19:29:04.259965 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.259946 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qvs82\"" Jun 02 19:29:04.336200 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.336148 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd3146f-a731-43f0-b236-394302ed8baf-tls-certs\") pod \"model-serving-api-6577596cd5-8wwnl\" (UID: \"7cd3146f-a731-43f0-b236-394302ed8baf\") " pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:04.336399 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.336254 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ln9n\" (UniqueName: \"kubernetes.io/projected/7cd3146f-a731-43f0-b236-394302ed8baf-kube-api-access-5ln9n\") pod \"model-serving-api-6577596cd5-8wwnl\" (UID: \"7cd3146f-a731-43f0-b236-394302ed8baf\") " pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:04.345150 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.345115 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:04.437035 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.436836 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd3146f-a731-43f0-b236-394302ed8baf-tls-certs\") pod \"model-serving-api-6577596cd5-8wwnl\" (UID: \"7cd3146f-a731-43f0-b236-394302ed8baf\") " pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:04.437035 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.436943 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5ln9n\" (UniqueName: \"kubernetes.io/projected/7cd3146f-a731-43f0-b236-394302ed8baf-kube-api-access-5ln9n\") pod \"model-serving-api-6577596cd5-8wwnl\" (UID: \"7cd3146f-a731-43f0-b236-394302ed8baf\") " pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:04.437331 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:29:04.437096 2552 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 02 19:29:04.437331 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:29:04.437177 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cd3146f-a731-43f0-b236-394302ed8baf-tls-certs podName:7cd3146f-a731-43f0-b236-394302ed8baf nodeName:}" failed. No retries permitted until 2026-06-02 19:29:04.93715474 +0000 UTC m=+475.027022957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7cd3146f-a731-43f0-b236-394302ed8baf-tls-certs") pod "model-serving-api-6577596cd5-8wwnl" (UID: "7cd3146f-a731-43f0-b236-394302ed8baf") : secret "model-serving-api-tls" not found Jun 02 19:29:04.449006 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.448967 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ln9n\" (UniqueName: \"kubernetes.io/projected/7cd3146f-a731-43f0-b236-394302ed8baf-kube-api-access-5ln9n\") pod \"model-serving-api-6577596cd5-8wwnl\" (UID: \"7cd3146f-a731-43f0-b236-394302ed8baf\") " pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:04.498769 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.498712 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-bd4dl"] Jun 02 19:29:04.501131 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:04.501103 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f23b73_2276_47ef_9c7b_0dc0910007f1.slice/crio-ff02c954a0f1ead8d19169d26bc2245c5ebae961638b755b8f2607f1cca7ffc3 WatchSource:0}: Error finding container ff02c954a0f1ead8d19169d26bc2245c5ebae961638b755b8f2607f1cca7ffc3: Status 404 returned error can't find the container with id ff02c954a0f1ead8d19169d26bc2245c5ebae961638b755b8f2607f1cca7ffc3 Jun 02 19:29:04.942727 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.942690 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd3146f-a731-43f0-b236-394302ed8baf-tls-certs\") pod \"model-serving-api-6577596cd5-8wwnl\" (UID: \"7cd3146f-a731-43f0-b236-394302ed8baf\") " pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:04.945328 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:04.945297 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd3146f-a731-43f0-b236-394302ed8baf-tls-certs\") pod \"model-serving-api-6577596cd5-8wwnl\" (UID: \"7cd3146f-a731-43f0-b236-394302ed8baf\") " pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:05.101458 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:05.101418 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-bd4dl" event={"ID":"37f23b73-2276-47ef-9c7b-0dc0910007f1","Type":"ContainerStarted","Data":"ff02c954a0f1ead8d19169d26bc2245c5ebae961638b755b8f2607f1cca7ffc3"} Jun 02 19:29:05.168826 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:05.168787 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:05.343021 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:05.342986 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-8wwnl"] Jun 02 19:29:05.345573 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:05.345537 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd3146f_a731_43f0_b236_394302ed8baf.slice/crio-236de7f01359d2b400c632f6b965d317856bd4a33499cc74310f21320b7b76c7 WatchSource:0}: Error finding container 236de7f01359d2b400c632f6b965d317856bd4a33499cc74310f21320b7b76c7: Status 404 returned error can't find the container with id 236de7f01359d2b400c632f6b965d317856bd4a33499cc74310f21320b7b76c7 Jun 02 19:29:06.108596 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:06.108553 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-8wwnl" event={"ID":"7cd3146f-a731-43f0-b236-394302ed8baf","Type":"ContainerStarted","Data":"236de7f01359d2b400c632f6b965d317856bd4a33499cc74310f21320b7b76c7"} Jun 02 19:29:09.121532 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:09.121480 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-bd4dl" event={"ID":"37f23b73-2276-47ef-9c7b-0dc0910007f1","Type":"ContainerStarted","Data":"c30e0b3f0c6b4374fd2504386dd140a62c953747bdbbb2cc711feed9f962ddf9"} Jun 02 19:29:09.122030 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:09.121616 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:09.123067 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:09.123037 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-8wwnl" event={"ID":"7cd3146f-a731-43f0-b236-394302ed8baf","Type":"ContainerStarted","Data":"c0fa2926bafa71374e4e99cd0ee63794bfca0472e8bf2a0547acc75af42ee048"} Jun 02 19:29:09.123199 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:09.123138 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:09.142166 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:09.142102 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-bd4dl" podStartSLOduration=2.182841015 podStartE2EDuration="6.142083667s" podCreationTimestamp="2026-06-02 19:29:03 +0000 UTC" firstStartedPulling="2026-06-02 19:29:04.502478252 +0000 UTC m=+474.592346464" lastFinishedPulling="2026-06-02 19:29:08.461720892 +0000 UTC m=+478.551589116" observedRunningTime="2026-06-02 19:29:09.140188962 +0000 UTC m=+479.230057197" watchObservedRunningTime="2026-06-02 19:29:09.142083667 +0000 UTC m=+479.231951970" Jun 02 19:29:09.159337 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:09.159274 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/model-serving-api-6577596cd5-8wwnl" podStartSLOduration=2.008062028 podStartE2EDuration="5.159255157s" podCreationTimestamp="2026-06-02 19:29:04 +0000 UTC" firstStartedPulling="2026-06-02 19:29:05.347951637 +0000 UTC m=+475.437819847" lastFinishedPulling="2026-06-02 19:29:08.499144754 +0000 UTC m=+478.589012976" observedRunningTime="2026-06-02 19:29:09.158117229 +0000 UTC m=+479.247985464" watchObservedRunningTime="2026-06-02 19:29:09.159255157 +0000 UTC m=+479.249123391" Jun 02 19:29:13.364583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.364542 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-k427x"] Jun 02 19:29:13.369367 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.369339 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:13.372700 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.372666 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 02 19:29:13.372834 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.372789 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hxjwp\"" Jun 02 19:29:13.378049 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.378023 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs"] Jun 02 19:29:13.383900 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.383871 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:13.391583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.391225 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 02 19:29:13.391723 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.391685 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wmrvl\"" Jun 02 19:29:13.395123 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.395093 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-k427x"] Jun 02 19:29:13.397813 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.397783 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs"] Jun 02 19:29:13.419665 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.419631 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l76gg\" (UniqueName: \"kubernetes.io/projected/05286ae9-7530-4264-928b-a175fd594e57-kube-api-access-l76gg\") pod \"kserve-controller-manager-7c99ffcdcd-k427x\" (UID: \"05286ae9-7530-4264-928b-a175fd594e57\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:13.419665 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.419680 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c44a5f1-62ae-4dc6-b425-86460cff13d1-cert\") pod \"llmisvc-controller-manager-c6b9847d7-kbnjs\" (UID: \"1c44a5f1-62ae-4dc6-b425-86460cff13d1\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:13.419933 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.419709 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zgvf\" (UniqueName: \"kubernetes.io/projected/1c44a5f1-62ae-4dc6-b425-86460cff13d1-kube-api-access-5zgvf\") pod \"llmisvc-controller-manager-c6b9847d7-kbnjs\" (UID: \"1c44a5f1-62ae-4dc6-b425-86460cff13d1\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:13.419933 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.419798 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05286ae9-7530-4264-928b-a175fd594e57-cert\") pod \"kserve-controller-manager-7c99ffcdcd-k427x\" (UID: \"05286ae9-7530-4264-928b-a175fd594e57\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:13.520934 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.520891 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5zgvf\" (UniqueName: \"kubernetes.io/projected/1c44a5f1-62ae-4dc6-b425-86460cff13d1-kube-api-access-5zgvf\") pod \"llmisvc-controller-manager-c6b9847d7-kbnjs\" (UID: \"1c44a5f1-62ae-4dc6-b425-86460cff13d1\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:13.521144 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.520948 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05286ae9-7530-4264-928b-a175fd594e57-cert\") pod \"kserve-controller-manager-7c99ffcdcd-k427x\" (UID: \"05286ae9-7530-4264-928b-a175fd594e57\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:13.521144 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.521019 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l76gg\" (UniqueName: \"kubernetes.io/projected/05286ae9-7530-4264-928b-a175fd594e57-kube-api-access-l76gg\") pod \"kserve-controller-manager-7c99ffcdcd-k427x\" (UID: \"05286ae9-7530-4264-928b-a175fd594e57\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:13.521144 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.521059 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c44a5f1-62ae-4dc6-b425-86460cff13d1-cert\") pod \"llmisvc-controller-manager-c6b9847d7-kbnjs\" (UID: \"1c44a5f1-62ae-4dc6-b425-86460cff13d1\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:13.521339 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:29:13.521161 2552 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 02 19:29:13.521339 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:29:13.521242 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c44a5f1-62ae-4dc6-b425-86460cff13d1-cert podName:1c44a5f1-62ae-4dc6-b425-86460cff13d1 nodeName:}" failed. No retries permitted until 2026-06-02 19:29:14.021220193 +0000 UTC m=+484.111088409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1c44a5f1-62ae-4dc6-b425-86460cff13d1-cert") pod "llmisvc-controller-manager-c6b9847d7-kbnjs" (UID: "1c44a5f1-62ae-4dc6-b425-86460cff13d1") : secret "llmisvc-webhook-server-cert" not found Jun 02 19:29:13.523447 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.523419 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05286ae9-7530-4264-928b-a175fd594e57-cert\") pod \"kserve-controller-manager-7c99ffcdcd-k427x\" (UID: \"05286ae9-7530-4264-928b-a175fd594e57\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:13.538926 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.538886 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l76gg\" (UniqueName: \"kubernetes.io/projected/05286ae9-7530-4264-928b-a175fd594e57-kube-api-access-l76gg\") pod \"kserve-controller-manager-7c99ffcdcd-k427x\" (UID: \"05286ae9-7530-4264-928b-a175fd594e57\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:13.539899 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.539878 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zgvf\" (UniqueName: \"kubernetes.io/projected/1c44a5f1-62ae-4dc6-b425-86460cff13d1-kube-api-access-5zgvf\") pod \"llmisvc-controller-manager-c6b9847d7-kbnjs\" (UID: \"1c44a5f1-62ae-4dc6-b425-86460cff13d1\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:13.682619 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.682530 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:13.845941 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.845910 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-k427x"] Jun 02 19:29:13.847312 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:13.847279 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05286ae9_7530_4264_928b_a175fd594e57.slice/crio-480e9627405a1608aacb68f8c2c45741f7477dc29c833aaab8d9985ff3364cf4 WatchSource:0}: Error finding container 480e9627405a1608aacb68f8c2c45741f7477dc29c833aaab8d9985ff3364cf4: Status 404 returned error can't find the container with id 480e9627405a1608aacb68f8c2c45741f7477dc29c833aaab8d9985ff3364cf4 Jun 02 19:29:13.977835 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.977727 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7"] Jun 02 19:29:13.982856 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.982829 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:13.988124 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.988098 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 19:29:13.989128 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.989105 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qd585\"" Jun 02 19:29:13.989128 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:13.989115 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 19:29:14.002257 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.001703 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7"] Jun 02 19:29:14.026698 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.026670 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.026892 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.026736 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c44a5f1-62ae-4dc6-b425-86460cff13d1-cert\") pod \"llmisvc-controller-manager-c6b9847d7-kbnjs\" (UID: \"1c44a5f1-62ae-4dc6-b425-86460cff13d1\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:14.026892 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.026817 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t69jm\" (UniqueName: \"kubernetes.io/projected/ba3d17c9-bdad-451a-8fe7-882aae4aed80-kube-api-access-t69jm\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.026892 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.026857 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.029225 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.029202 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c44a5f1-62ae-4dc6-b425-86460cff13d1-cert\") pod \"llmisvc-controller-manager-c6b9847d7-kbnjs\" (UID: \"1c44a5f1-62ae-4dc6-b425-86460cff13d1\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:14.049580 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.049543 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2"] Jun 02 19:29:14.053094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.053071 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.056096 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.056073 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 19:29:14.056240 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.056160 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jbl77\"" Jun 02 19:29:14.056240 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.056227 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 19:29:14.068553 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.068522 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2"] Jun 02 19:29:14.127838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.127802 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.128051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.127871 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t69jm\" (UniqueName: \"kubernetes.io/projected/ba3d17c9-bdad-451a-8fe7-882aae4aed80-kube-api-access-t69jm\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.128051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.127895 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6785b6d-7b5b-4358-bf79-fbd38042ab29-tmp\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.128051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.127913 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnc4z\" (UniqueName: \"kubernetes.io/projected/c6785b6d-7b5b-4358-bf79-fbd38042ab29-kube-api-access-rnc4z\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.128051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.127933 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.128051 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.128012 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6785b6d-7b5b-4358-bf79-fbd38042ab29-tls-certs\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.128265 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.128246 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.128304 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.128289 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.139784 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.139759 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t69jm\" (UniqueName: \"kubernetes.io/projected/ba3d17c9-bdad-451a-8fe7-882aae4aed80-kube-api-access-t69jm\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.142439 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.142404 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" event={"ID":"05286ae9-7530-4264-928b-a175fd594e57","Type":"ContainerStarted","Data":"480e9627405a1608aacb68f8c2c45741f7477dc29c833aaab8d9985ff3364cf4"} Jun 02 19:29:14.228878 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.228777 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6785b6d-7b5b-4358-bf79-fbd38042ab29-tmp\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.228878 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.228820 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rnc4z\" (UniqueName: \"kubernetes.io/projected/c6785b6d-7b5b-4358-bf79-fbd38042ab29-kube-api-access-rnc4z\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.229108 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.228882 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6785b6d-7b5b-4358-bf79-fbd38042ab29-tls-certs\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.231167 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.231139 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6785b6d-7b5b-4358-bf79-fbd38042ab29-tmp\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.231409 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.231390 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6785b6d-7b5b-4358-bf79-fbd38042ab29-tls-certs\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.236936 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.236915 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnc4z\" (UniqueName: \"kubernetes.io/projected/c6785b6d-7b5b-4358-bf79-fbd38042ab29-kube-api-access-rnc4z\") pod \"kube-auth-proxy-897b5b6bb-74tz2\" (UID: \"c6785b6d-7b5b-4358-bf79-fbd38042ab29\") " pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.292588 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.292550 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:14.301372 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.301341 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:14.363710 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.363674 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" Jun 02 19:29:14.451203 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.451172 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7"] Jun 02 19:29:14.455062 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:14.455036 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba3d17c9_bdad_451a_8fe7_882aae4aed80.slice/crio-72c5b418f46871be3d24d04c96e0f438642b90111c22822154574eaddfd20d04 WatchSource:0}: Error finding container 72c5b418f46871be3d24d04c96e0f438642b90111c22822154574eaddfd20d04: Status 404 returned error can't find the container with id 72c5b418f46871be3d24d04c96e0f438642b90111c22822154574eaddfd20d04 Jun 02 19:29:14.476318 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.476259 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs"] Jun 02 19:29:14.480946 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:14.480555 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c44a5f1_62ae_4dc6_b425_86460cff13d1.slice/crio-3ac9504f5d9e076ee573cf39e384692a58271b205f5ece7662512f360b8a9142 WatchSource:0}: Error finding container 3ac9504f5d9e076ee573cf39e384692a58271b205f5ece7662512f360b8a9142: Status 404 returned error can't find the container with id 3ac9504f5d9e076ee573cf39e384692a58271b205f5ece7662512f360b8a9142 Jun 02 19:29:14.541452 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:14.541410 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2"] Jun 02 19:29:14.542658 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:14.542632 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6785b6d_7b5b_4358_bf79_fbd38042ab29.slice/crio-fb83e8f1aca1e21d13965bea28f9ad0ca5b81c6eac0837b35173860eec9fa66a WatchSource:0}: Error finding container fb83e8f1aca1e21d13965bea28f9ad0ca5b81c6eac0837b35173860eec9fa66a: Status 404 returned error can't find the container with id fb83e8f1aca1e21d13965bea28f9ad0ca5b81c6eac0837b35173860eec9fa66a Jun 02 19:29:15.148553 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:15.148516 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" event={"ID":"c6785b6d-7b5b-4358-bf79-fbd38042ab29","Type":"ContainerStarted","Data":"fb83e8f1aca1e21d13965bea28f9ad0ca5b81c6eac0837b35173860eec9fa66a"} Jun 02 19:29:15.149786 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:15.149759 2552 generic.go:358] "Generic (PLEG): container finished" podID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerID="b54862f677f2dc1cfb6a8000a5b80aec0da6d6c7d09bbe467f139c83458ffdee" exitCode=0 Jun 02 19:29:15.149928 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:15.149812 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" event={"ID":"ba3d17c9-bdad-451a-8fe7-882aae4aed80","Type":"ContainerDied","Data":"b54862f677f2dc1cfb6a8000a5b80aec0da6d6c7d09bbe467f139c83458ffdee"} Jun 02 19:29:15.149928 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:15.149844 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" event={"ID":"ba3d17c9-bdad-451a-8fe7-882aae4aed80","Type":"ContainerStarted","Data":"72c5b418f46871be3d24d04c96e0f438642b90111c22822154574eaddfd20d04"} Jun 02 19:29:15.150924 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:15.150853 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" event={"ID":"1c44a5f1-62ae-4dc6-b425-86460cff13d1","Type":"ContainerStarted","Data":"3ac9504f5d9e076ee573cf39e384692a58271b205f5ece7662512f360b8a9142"} Jun 02 19:29:17.162417 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:17.162372 2552 generic.go:358] "Generic (PLEG): container finished" podID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerID="c34dcc69e7bb494756a99516daad16fbfeb15288ab1d83db40a8dedd0e1323ff" exitCode=0 Jun 02 19:29:17.162913 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:17.162461 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" event={"ID":"ba3d17c9-bdad-451a-8fe7-882aae4aed80","Type":"ContainerDied","Data":"c34dcc69e7bb494756a99516daad16fbfeb15288ab1d83db40a8dedd0e1323ff"} Jun 02 19:29:18.168384 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:18.168344 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" event={"ID":"c6785b6d-7b5b-4358-bf79-fbd38042ab29","Type":"ContainerStarted","Data":"e0e4422d00b6ea7d12d4b56ea2f5e28634cd3c307aaa4c852445e6b47df164cf"} Jun 02 19:29:18.170161 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:18.170127 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" event={"ID":"05286ae9-7530-4264-928b-a175fd594e57","Type":"ContainerStarted","Data":"896c18f320a2b82fb19a5245a042bfec176c8357deb1499e143ec6986b834955"} Jun 02 19:29:18.170316 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:18.170263 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:18.172386 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:18.172357 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" event={"ID":"ba3d17c9-bdad-451a-8fe7-882aae4aed80","Type":"ContainerStarted","Data":"d84e4271c078455ebc973645a5b7aa3d0f12f82856681a38a0dd144786f9633e"} Jun 02 19:29:18.186821 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:18.186760 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-897b5b6bb-74tz2" podStartSLOduration=0.989267621 podStartE2EDuration="4.186743265s" podCreationTimestamp="2026-06-02 19:29:14 +0000 UTC" firstStartedPulling="2026-06-02 19:29:14.545082309 +0000 UTC m=+484.634950523" lastFinishedPulling="2026-06-02 19:29:17.742557957 +0000 UTC m=+487.832426167" observedRunningTime="2026-06-02 19:29:18.18487221 +0000 UTC m=+488.274740442" watchObservedRunningTime="2026-06-02 19:29:18.186743265 +0000 UTC m=+488.276611500" Jun 02 19:29:18.202876 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:18.201108 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" podStartSLOduration=1.309320356 podStartE2EDuration="5.201088097s" podCreationTimestamp="2026-06-02 19:29:13 +0000 UTC" firstStartedPulling="2026-06-02 19:29:13.848688939 +0000 UTC m=+483.938557153" lastFinishedPulling="2026-06-02 19:29:17.74045667 +0000 UTC m=+487.830324894" observedRunningTime="2026-06-02 19:29:18.200875484 +0000 UTC m=+488.290743755" watchObservedRunningTime="2026-06-02 19:29:18.201088097 +0000 UTC m=+488.290956330" Jun 02 19:29:18.221711 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:18.221645 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" podStartSLOduration=4.290823431 podStartE2EDuration="5.221621338s" podCreationTimestamp="2026-06-02 19:29:13 +0000 UTC" firstStartedPulling="2026-06-02 19:29:15.150954943 +0000 UTC m=+485.240823171" lastFinishedPulling="2026-06-02 19:29:16.081752852 +0000 UTC m=+486.171621078" observedRunningTime="2026-06-02 19:29:18.219740102 +0000 UTC m=+488.309608336" watchObservedRunningTime="2026-06-02 19:29:18.221621338 +0000 UTC m=+488.311489572" Jun 02 19:29:19.178481 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:19.178441 2552 generic.go:358] "Generic (PLEG): container finished" podID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerID="d84e4271c078455ebc973645a5b7aa3d0f12f82856681a38a0dd144786f9633e" exitCode=0 Jun 02 19:29:19.178966 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:19.178537 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" event={"ID":"ba3d17c9-bdad-451a-8fe7-882aae4aed80","Type":"ContainerDied","Data":"d84e4271c078455ebc973645a5b7aa3d0f12f82856681a38a0dd144786f9633e"} Jun 02 19:29:19.179964 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:19.179934 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" event={"ID":"1c44a5f1-62ae-4dc6-b425-86460cff13d1","Type":"ContainerStarted","Data":"ef8a9bc411057d83539d95dbfa55e6af1832a55340d6fd73619f3cbeba73b523"} Jun 02 19:29:19.214667 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:19.214599 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" podStartSLOduration=2.175696092 podStartE2EDuration="6.214578949s" podCreationTimestamp="2026-06-02 19:29:13 +0000 UTC" firstStartedPulling="2026-06-02 19:29:14.482461117 +0000 UTC m=+484.572329330" lastFinishedPulling="2026-06-02 19:29:18.521343959 +0000 UTC m=+488.611212187" observedRunningTime="2026-06-02 19:29:19.213165814 +0000 UTC m=+489.303034045" watchObservedRunningTime="2026-06-02 19:29:19.214578949 +0000 UTC m=+489.304447186" Jun 02 19:29:20.130379 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.130350 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-bd4dl" Jun 02 19:29:20.132976 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.132952 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-6577596cd5-8wwnl" Jun 02 19:29:20.195825 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.195793 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:20.335187 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.335156 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:20.393805 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.393707 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t69jm\" (UniqueName: \"kubernetes.io/projected/ba3d17c9-bdad-451a-8fe7-882aae4aed80-kube-api-access-t69jm\") pod \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " Jun 02 19:29:20.393805 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.393786 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-bundle\") pod \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " Jun 02 19:29:20.394055 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.393842 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-util\") pod \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\" (UID: \"ba3d17c9-bdad-451a-8fe7-882aae4aed80\") " Jun 02 19:29:20.397059 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.396957 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-bundle" (OuterVolumeSpecName: "bundle") pod "ba3d17c9-bdad-451a-8fe7-882aae4aed80" (UID: "ba3d17c9-bdad-451a-8fe7-882aae4aed80"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:29:20.398473 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.398415 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba3d17c9-bdad-451a-8fe7-882aae4aed80-kube-api-access-t69jm" (OuterVolumeSpecName: "kube-api-access-t69jm") pod "ba3d17c9-bdad-451a-8fe7-882aae4aed80" (UID: "ba3d17c9-bdad-451a-8fe7-882aae4aed80"). InnerVolumeSpecName "kube-api-access-t69jm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:29:20.495381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.495344 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t69jm\" (UniqueName: \"kubernetes.io/projected/ba3d17c9-bdad-451a-8fe7-882aae4aed80-kube-api-access-t69jm\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:29:20.495381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.495376 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:29:20.632332 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.632240 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-util" (OuterVolumeSpecName: "util") pod "ba3d17c9-bdad-451a-8fe7-882aae4aed80" (UID: "ba3d17c9-bdad-451a-8fe7-882aae4aed80"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:29:20.697056 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:20.696966 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3d17c9-bdad-451a-8fe7-882aae4aed80-util\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:29:21.200647 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:21.200618 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" Jun 02 19:29:21.201139 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:21.200608 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835gm6p7" event={"ID":"ba3d17c9-bdad-451a-8fe7-882aae4aed80","Type":"ContainerDied","Data":"72c5b418f46871be3d24d04c96e0f438642b90111c22822154574eaddfd20d04"} Jun 02 19:29:21.201139 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:21.200725 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72c5b418f46871be3d24d04c96e0f438642b90111c22822154574eaddfd20d04" Jun 02 19:29:28.359625 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.359583 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw"] Jun 02 19:29:28.360127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.360006 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerName="util" Jun 02 19:29:28.360127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.360021 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerName="util" Jun 02 19:29:28.360127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.360052 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerName="pull" Jun 02 19:29:28.360127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.360058 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerName="pull" Jun 02 19:29:28.360127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.360065 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerName="extract" Jun 02 19:29:28.360127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.360070 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerName="extract" Jun 02 19:29:28.360336 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.360142 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="ba3d17c9-bdad-451a-8fe7-882aae4aed80" containerName="extract" Jun 02 19:29:28.363606 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.363588 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.366934 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.366908 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 19:29:28.367089 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.366970 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 19:29:28.367868 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.367854 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qd585\"" Jun 02 19:29:28.376968 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.376943 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw"] Jun 02 19:29:28.470117 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.470070 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.470305 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.470131 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.470305 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.470212 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv256\" (UniqueName: \"kubernetes.io/projected/587a2fb0-e36d-4000-8903-6b433dac1b7f-kube-api-access-sv256\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.571187 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.571147 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.571377 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.571217 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sv256\" (UniqueName: \"kubernetes.io/projected/587a2fb0-e36d-4000-8903-6b433dac1b7f-kube-api-access-sv256\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.571377 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.571302 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.571674 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.571651 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.571741 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.571726 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.580110 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.580083 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv256\" (UniqueName: \"kubernetes.io/projected/587a2fb0-e36d-4000-8903-6b433dac1b7f-kube-api-access-sv256\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.673207 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.673103 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:28.829194 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:28.829169 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw"] Jun 02 19:29:28.831061 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:28.831020 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod587a2fb0_e36d_4000_8903_6b433dac1b7f.slice/crio-39eba37d1f3beae2a0b67dc80951bef8f62c2849fd6036b381c562834e9b4e3e WatchSource:0}: Error finding container 39eba37d1f3beae2a0b67dc80951bef8f62c2849fd6036b381c562834e9b4e3e: Status 404 returned error can't find the container with id 39eba37d1f3beae2a0b67dc80951bef8f62c2849fd6036b381c562834e9b4e3e Jun 02 19:29:29.230779 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.230691 2552 generic.go:358] "Generic (PLEG): container finished" podID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerID="7178d81cba34f37ddde7dab8c03364586aa5ca3f93465a7e55d822e836853a22" exitCode=0 Jun 02 19:29:29.230928 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.230781 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" event={"ID":"587a2fb0-e36d-4000-8903-6b433dac1b7f","Type":"ContainerDied","Data":"7178d81cba34f37ddde7dab8c03364586aa5ca3f93465a7e55d822e836853a22"} Jun 02 19:29:29.230928 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.230819 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" event={"ID":"587a2fb0-e36d-4000-8903-6b433dac1b7f","Type":"ContainerStarted","Data":"39eba37d1f3beae2a0b67dc80951bef8f62c2849fd6036b381c562834e9b4e3e"} Jun 02 19:29:29.465175 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.465134 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-79dcdb775-lqbbc"] Jun 02 19:29:29.469371 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.469349 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" Jun 02 19:29:29.473684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.473653 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 02 19:29:29.473684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.473668 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ftjcx\"" Jun 02 19:29:29.479677 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.479645 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-79dcdb775-lqbbc"] Jun 02 19:29:29.580136 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.580095 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4fzk\" (UniqueName: \"kubernetes.io/projected/a4c79d26-173e-4ae9-abf6-c493746cdf25-kube-api-access-l4fzk\") pod \"dashboard-redirect-79dcdb775-lqbbc\" (UID: \"a4c79d26-173e-4ae9-abf6-c493746cdf25\") " pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" Jun 02 19:29:29.580136 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.580137 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a4c79d26-173e-4ae9-abf6-c493746cdf25-redirect-config\") pod \"dashboard-redirect-79dcdb775-lqbbc\" (UID: \"a4c79d26-173e-4ae9-abf6-c493746cdf25\") " pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" Jun 02 19:29:29.597044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.597007 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7"] Jun 02 19:29:29.600422 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.600402 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:29.603262 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.603230 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-c8bcf\"" Jun 02 19:29:29.603262 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.603230 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 02 19:29:29.603417 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.603262 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 02 19:29:29.615761 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.615736 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7"] Jun 02 19:29:29.681465 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.681427 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l4fzk\" (UniqueName: \"kubernetes.io/projected/a4c79d26-173e-4ae9-abf6-c493746cdf25-kube-api-access-l4fzk\") pod \"dashboard-redirect-79dcdb775-lqbbc\" (UID: \"a4c79d26-173e-4ae9-abf6-c493746cdf25\") " pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" Jun 02 19:29:29.681465 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.681472 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a4c79d26-173e-4ae9-abf6-c493746cdf25-redirect-config\") pod \"dashboard-redirect-79dcdb775-lqbbc\" (UID: \"a4c79d26-173e-4ae9-abf6-c493746cdf25\") " pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" Jun 02 19:29:29.681719 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.681534 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qczl4\" (UniqueName: \"kubernetes.io/projected/c519e704-e20d-46de-a724-21f22c29f6e8-kube-api-access-qczl4\") pod \"servicemesh-operator3-55f49c5f94-dtnw7\" (UID: \"c519e704-e20d-46de-a724-21f22c29f6e8\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:29.681719 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.681573 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c519e704-e20d-46de-a724-21f22c29f6e8-operator-config\") pod \"servicemesh-operator3-55f49c5f94-dtnw7\" (UID: \"c519e704-e20d-46de-a724-21f22c29f6e8\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:29.682215 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.682195 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/a4c79d26-173e-4ae9-abf6-c493746cdf25-redirect-config\") pod \"dashboard-redirect-79dcdb775-lqbbc\" (UID: \"a4c79d26-173e-4ae9-abf6-c493746cdf25\") " pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" Jun 02 19:29:29.691075 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.691048 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4fzk\" (UniqueName: \"kubernetes.io/projected/a4c79d26-173e-4ae9-abf6-c493746cdf25-kube-api-access-l4fzk\") pod \"dashboard-redirect-79dcdb775-lqbbc\" (UID: \"a4c79d26-173e-4ae9-abf6-c493746cdf25\") " pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" Jun 02 19:29:29.781596 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.781557 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" Jun 02 19:29:29.782735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.782451 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qczl4\" (UniqueName: \"kubernetes.io/projected/c519e704-e20d-46de-a724-21f22c29f6e8-kube-api-access-qczl4\") pod \"servicemesh-operator3-55f49c5f94-dtnw7\" (UID: \"c519e704-e20d-46de-a724-21f22c29f6e8\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:29.782735 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.782526 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c519e704-e20d-46de-a724-21f22c29f6e8-operator-config\") pod \"servicemesh-operator3-55f49c5f94-dtnw7\" (UID: \"c519e704-e20d-46de-a724-21f22c29f6e8\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:29.785856 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.785791 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c519e704-e20d-46de-a724-21f22c29f6e8-operator-config\") pod \"servicemesh-operator3-55f49c5f94-dtnw7\" (UID: \"c519e704-e20d-46de-a724-21f22c29f6e8\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:29.797826 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.797794 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qczl4\" (UniqueName: \"kubernetes.io/projected/c519e704-e20d-46de-a724-21f22c29f6e8-kube-api-access-qczl4\") pod \"servicemesh-operator3-55f49c5f94-dtnw7\" (UID: \"c519e704-e20d-46de-a724-21f22c29f6e8\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:29.910204 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.910173 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:29.945044 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:29.944953 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-79dcdb775-lqbbc"] Jun 02 19:29:29.947289 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:29.947247 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c79d26_173e_4ae9_abf6_c493746cdf25.slice/crio-0c1c26d202cb44dcdaf3ba15a510b5272efda30859dfc4810b0546384f8baeac WatchSource:0}: Error finding container 0c1c26d202cb44dcdaf3ba15a510b5272efda30859dfc4810b0546384f8baeac: Status 404 returned error can't find the container with id 0c1c26d202cb44dcdaf3ba15a510b5272efda30859dfc4810b0546384f8baeac Jun 02 19:29:30.062864 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:30.062835 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7"] Jun 02 19:29:30.064803 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:30.064773 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc519e704_e20d_46de_a724_21f22c29f6e8.slice/crio-f3938bcba7e1254a09cf7028e313156ec08d46d8b37e0561dfe17bca5d8f0c93 WatchSource:0}: Error finding container f3938bcba7e1254a09cf7028e313156ec08d46d8b37e0561dfe17bca5d8f0c93: Status 404 returned error can't find the container with id f3938bcba7e1254a09cf7028e313156ec08d46d8b37e0561dfe17bca5d8f0c93 Jun 02 19:29:30.243906 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:30.243794 2552 generic.go:358] "Generic (PLEG): container finished" podID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerID="864cfa3be6c1bb768fd26e5f6d25608cb15d33ff05cc2f3a13d85e505440e7e0" exitCode=0 Jun 02 19:29:30.243906 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:30.243881 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" event={"ID":"587a2fb0-e36d-4000-8903-6b433dac1b7f","Type":"ContainerDied","Data":"864cfa3be6c1bb768fd26e5f6d25608cb15d33ff05cc2f3a13d85e505440e7e0"} Jun 02 19:29:30.245098 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:30.245071 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" event={"ID":"a4c79d26-173e-4ae9-abf6-c493746cdf25","Type":"ContainerStarted","Data":"0c1c26d202cb44dcdaf3ba15a510b5272efda30859dfc4810b0546384f8baeac"} Jun 02 19:29:30.246385 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:30.246363 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" event={"ID":"c519e704-e20d-46de-a724-21f22c29f6e8","Type":"ContainerStarted","Data":"f3938bcba7e1254a09cf7028e313156ec08d46d8b37e0561dfe17bca5d8f0c93"} Jun 02 19:29:31.254419 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:31.254330 2552 generic.go:358] "Generic (PLEG): container finished" podID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerID="97291af932706e5bb267466c748a3d60b161e97e958986d6a42055c9210c2c31" exitCode=0 Jun 02 19:29:31.254419 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:31.254414 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" event={"ID":"587a2fb0-e36d-4000-8903-6b433dac1b7f","Type":"ContainerDied","Data":"97291af932706e5bb267466c748a3d60b161e97e958986d6a42055c9210c2c31"} Jun 02 19:29:33.130825 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.130798 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:33.218840 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.218807 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-util\") pod \"587a2fb0-e36d-4000-8903-6b433dac1b7f\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " Jun 02 19:29:33.219004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.218866 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv256\" (UniqueName: \"kubernetes.io/projected/587a2fb0-e36d-4000-8903-6b433dac1b7f-kube-api-access-sv256\") pod \"587a2fb0-e36d-4000-8903-6b433dac1b7f\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " Jun 02 19:29:33.219004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.218928 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-bundle\") pod \"587a2fb0-e36d-4000-8903-6b433dac1b7f\" (UID: \"587a2fb0-e36d-4000-8903-6b433dac1b7f\") " Jun 02 19:29:33.220251 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.220203 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-bundle" (OuterVolumeSpecName: "bundle") pod "587a2fb0-e36d-4000-8903-6b433dac1b7f" (UID: "587a2fb0-e36d-4000-8903-6b433dac1b7f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:29:33.221613 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.221575 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/587a2fb0-e36d-4000-8903-6b433dac1b7f-kube-api-access-sv256" (OuterVolumeSpecName: "kube-api-access-sv256") pod "587a2fb0-e36d-4000-8903-6b433dac1b7f" (UID: "587a2fb0-e36d-4000-8903-6b433dac1b7f"). InnerVolumeSpecName "kube-api-access-sv256". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:29:33.225109 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.225082 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-util" (OuterVolumeSpecName: "util") pod "587a2fb0-e36d-4000-8903-6b433dac1b7f" (UID: "587a2fb0-e36d-4000-8903-6b433dac1b7f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:29:33.267898 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.267841 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" event={"ID":"c519e704-e20d-46de-a724-21f22c29f6e8","Type":"ContainerStarted","Data":"8b3aad1707a9cfa24737a66fd6ff83263a42d17fe4c4cbadee062f393c843752"} Jun 02 19:29:33.267898 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.267899 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:33.269692 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.269664 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" event={"ID":"587a2fb0-e36d-4000-8903-6b433dac1b7f","Type":"ContainerDied","Data":"39eba37d1f3beae2a0b67dc80951bef8f62c2849fd6036b381c562834e9b4e3e"} Jun 02 19:29:33.269878 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.269858 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39eba37d1f3beae2a0b67dc80951bef8f62c2849fd6036b381c562834e9b4e3e" Jun 02 19:29:33.269878 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.269680 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avn8vw" Jun 02 19:29:33.271792 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.271763 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" event={"ID":"a4c79d26-173e-4ae9-abf6-c493746cdf25","Type":"ContainerStarted","Data":"fd7f7faab9451c1005cf8293bb864171d34d03b779743abbd87e7036617d7a9f"} Jun 02 19:29:33.291727 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.291658 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" podStartSLOduration=1.190831173 podStartE2EDuration="4.291637525s" podCreationTimestamp="2026-06-02 19:29:29 +0000 UTC" firstStartedPulling="2026-06-02 19:29:30.067343346 +0000 UTC m=+500.157211557" lastFinishedPulling="2026-06-02 19:29:33.168149697 +0000 UTC m=+503.258017909" observedRunningTime="2026-06-02 19:29:33.288736693 +0000 UTC m=+503.378604928" watchObservedRunningTime="2026-06-02 19:29:33.291637525 +0000 UTC m=+503.381505760" Jun 02 19:29:33.305592 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.305535 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-79dcdb775-lqbbc" podStartSLOduration=1.092016842 podStartE2EDuration="4.305519402s" podCreationTimestamp="2026-06-02 19:29:29 +0000 UTC" firstStartedPulling="2026-06-02 19:29:29.94884057 +0000 UTC m=+500.038708784" lastFinishedPulling="2026-06-02 19:29:33.162343119 +0000 UTC m=+503.252211344" observedRunningTime="2026-06-02 19:29:33.304518195 +0000 UTC m=+503.394386430" watchObservedRunningTime="2026-06-02 19:29:33.305519402 +0000 UTC m=+503.395387634" Jun 02 19:29:33.320430 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.320390 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-util\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:29:33.320430 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.320432 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv256\" (UniqueName: \"kubernetes.io/projected/587a2fb0-e36d-4000-8903-6b433dac1b7f-kube-api-access-sv256\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:29:33.320902 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:33.320451 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/587a2fb0-e36d-4000-8903-6b433dac1b7f-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:29:36.419079 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.419037 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b578888bd-zx59h"] Jun 02 19:29:36.419785 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.419761 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerName="pull" Jun 02 19:29:36.419901 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.419802 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerName="pull" Jun 02 19:29:36.419901 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.419816 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerName="util" Jun 02 19:29:36.419901 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.419823 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerName="util" Jun 02 19:29:36.419901 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.419833 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerName="extract" Jun 02 19:29:36.419901 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.419841 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerName="extract" Jun 02 19:29:36.420125 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.419970 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="587a2fb0-e36d-4000-8903-6b433dac1b7f" containerName="extract" Jun 02 19:29:36.424323 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.424298 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.434180 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.434151 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b578888bd-zx59h"] Jun 02 19:29:36.552542 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.552475 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-serving-cert\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.552757 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.552546 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-trusted-ca-bundle\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.552757 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.552649 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-oauth-config\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.552757 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.552685 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-service-ca\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.552757 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.552710 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtlkn\" (UniqueName: \"kubernetes.io/projected/2f233028-e8a5-4cbb-9159-44ecf7e41948-kube-api-access-jtlkn\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.552757 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.552755 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-config\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.553009 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.552854 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-oauth-serving-cert\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.654034 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.653988 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-serving-cert\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.654034 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.654038 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-trusted-ca-bundle\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.654291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.654095 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-oauth-config\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.654291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.654116 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-service-ca\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.654291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.654134 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlkn\" (UniqueName: \"kubernetes.io/projected/2f233028-e8a5-4cbb-9159-44ecf7e41948-kube-api-access-jtlkn\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.654291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.654174 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-config\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.654291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.654230 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-oauth-serving-cert\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.655117 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.655086 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-config\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.655237 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.655187 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-service-ca\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.655237 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.655187 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-oauth-serving-cert\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.655376 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.655357 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f233028-e8a5-4cbb-9159-44ecf7e41948-trusted-ca-bundle\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.656618 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.656587 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-oauth-config\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.656795 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.656774 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f233028-e8a5-4cbb-9159-44ecf7e41948-console-serving-cert\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.662682 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.662661 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlkn\" (UniqueName: \"kubernetes.io/projected/2f233028-e8a5-4cbb-9159-44ecf7e41948-kube-api-access-jtlkn\") pod \"console-7b578888bd-zx59h\" (UID: \"2f233028-e8a5-4cbb-9159-44ecf7e41948\") " pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.736838 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.736735 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:36.876997 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:36.876968 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b578888bd-zx59h"] Jun 02 19:29:36.879262 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:36.879231 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f233028_e8a5_4cbb_9159_44ecf7e41948.slice/crio-27b103e07723f16f6c62fb9a3c55ebaf76798980fb6ac67f43df6855000c27af WatchSource:0}: Error finding container 27b103e07723f16f6c62fb9a3c55ebaf76798980fb6ac67f43df6855000c27af: Status 404 returned error can't find the container with id 27b103e07723f16f6c62fb9a3c55ebaf76798980fb6ac67f43df6855000c27af Jun 02 19:29:37.289595 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:37.289555 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b578888bd-zx59h" event={"ID":"2f233028-e8a5-4cbb-9159-44ecf7e41948","Type":"ContainerStarted","Data":"f66cce7f6677be395de37624ec92dc650164b6698e665ece4a97a6b070097cec"} Jun 02 19:29:37.289595 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:37.289600 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b578888bd-zx59h" event={"ID":"2f233028-e8a5-4cbb-9159-44ecf7e41948","Type":"ContainerStarted","Data":"27b103e07723f16f6c62fb9a3c55ebaf76798980fb6ac67f43df6855000c27af"} Jun 02 19:29:37.308314 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:37.307978 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7b578888bd-zx59h" podStartSLOduration=1.307960489 podStartE2EDuration="1.307960489s" podCreationTimestamp="2026-06-02 19:29:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:29:37.30794323 +0000 UTC m=+507.397811463" watchObservedRunningTime="2026-06-02 19:29:37.307960489 +0000 UTC m=+507.397828724" Jun 02 19:29:44.278674 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:44.278637 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dtnw7" Jun 02 19:29:46.002994 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.002954 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh"] Jun 02 19:29:46.012393 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.012368 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.016085 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.016055 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 02 19:29:46.016357 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.016339 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 02 19:29:46.016357 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.016352 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rl5f9\"" Jun 02 19:29:46.016610 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.016590 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 02 19:29:46.016685 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.016609 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 02 19:29:46.025517 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.025471 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh"] Jun 02 19:29:46.041768 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.041738 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.041935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.041849 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.041935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.041884 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fde924f-ad81-40f6-b877-7d36e75e90c8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.041935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.041905 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.041935 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.041927 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.042073 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.042004 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.042073 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.042036 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnxkj\" (UniqueName: \"kubernetes.io/projected/7fde924f-ad81-40f6-b877-7d36e75e90c8-kube-api-access-pnxkj\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.142465 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.142426 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.142465 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.142468 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fde924f-ad81-40f6-b877-7d36e75e90c8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.142746 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.142488 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.142746 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.142538 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.142746 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.142564 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.142746 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.142595 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pnxkj\" (UniqueName: \"kubernetes.io/projected/7fde924f-ad81-40f6-b877-7d36e75e90c8-kube-api-access-pnxkj\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.142746 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.142659 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.143390 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.143248 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.145292 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.145264 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fde924f-ad81-40f6-b877-7d36e75e90c8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.145466 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.145449 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.145638 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.145614 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.145702 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.145647 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.169967 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.169934 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnxkj\" (UniqueName: \"kubernetes.io/projected/7fde924f-ad81-40f6-b877-7d36e75e90c8-kube-api-access-pnxkj\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.170242 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.170217 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fde924f-ad81-40f6-b877-7d36e75e90c8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-xwqhh\" (UID: \"7fde924f-ad81-40f6-b877-7d36e75e90c8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.322395 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.322367 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:46.469881 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.469845 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh"] Jun 02 19:29:46.471768 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:29:46.471739 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fde924f_ad81_40f6_b877_7d36e75e90c8.slice/crio-92dba1dd056b68cce41d18991d0037423ee4bc34ba4539697be4a23fc134eaf7 WatchSource:0}: Error finding container 92dba1dd056b68cce41d18991d0037423ee4bc34ba4539697be4a23fc134eaf7: Status 404 returned error can't find the container with id 92dba1dd056b68cce41d18991d0037423ee4bc34ba4539697be4a23fc134eaf7 Jun 02 19:29:46.737448 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.737341 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:46.737448 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.737399 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:46.742557 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:46.742526 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:47.330139 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:47.330085 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" event={"ID":"7fde924f-ad81-40f6-b877-7d36e75e90c8","Type":"ContainerStarted","Data":"92dba1dd056b68cce41d18991d0037423ee4bc34ba4539697be4a23fc134eaf7"} Jun 02 19:29:47.335195 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:47.335164 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b578888bd-zx59h" Jun 02 19:29:47.384478 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:47.384437 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-795c9c8864-9trx6"] Jun 02 19:29:49.121114 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:49.121067 2552 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 02 19:29:49.121405 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:49.121159 2552 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 02 19:29:49.185648 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:49.185622 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-k427x" Jun 02 19:29:49.340942 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:49.340896 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" event={"ID":"7fde924f-ad81-40f6-b877-7d36e75e90c8","Type":"ContainerStarted","Data":"5fbe23826406aacb51a7e62f019b7e42e2b964841420c81f9bf3eb8cb4b27227"} Jun 02 19:29:49.341127 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:49.341086 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:29:49.362427 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:49.362358 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" podStartSLOduration=1.715554339 podStartE2EDuration="4.362338877s" podCreationTimestamp="2026-06-02 19:29:45 +0000 UTC" firstStartedPulling="2026-06-02 19:29:46.473945073 +0000 UTC m=+516.563813285" lastFinishedPulling="2026-06-02 19:29:49.120729597 +0000 UTC m=+519.210597823" observedRunningTime="2026-06-02 19:29:49.359572057 +0000 UTC m=+519.449440296" watchObservedRunningTime="2026-06-02 19:29:49.362338877 +0000 UTC m=+519.452207111" Jun 02 19:29:50.347376 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:50.347342 2552 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-xwqhh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 02 19:29:50.347798 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:50.347403 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" podUID="7fde924f-ad81-40f6-b877-7d36e75e90c8" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 19:29:51.203110 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:51.203080 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-kbnjs" Jun 02 19:29:53.346487 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:29:53.346453 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-xwqhh" Jun 02 19:30:12.422204 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.422105 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-795c9c8864-9trx6" podUID="19e20106-ec71-4202-aabd-914e728ac6de" containerName="console" containerID="cri-o://7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1" gracePeriod=15 Jun 02 19:30:12.687164 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.687136 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-795c9c8864-9trx6_19e20106-ec71-4202-aabd-914e728ac6de/console/0.log" Jun 02 19:30:12.687310 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.687207 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:30:12.813352 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813315 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-service-ca\") pod \"19e20106-ec71-4202-aabd-914e728ac6de\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " Jun 02 19:30:12.813542 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813407 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxkgq\" (UniqueName: \"kubernetes.io/projected/19e20106-ec71-4202-aabd-914e728ac6de-kube-api-access-fxkgq\") pod \"19e20106-ec71-4202-aabd-914e728ac6de\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " Jun 02 19:30:12.813592 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813538 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-oauth-serving-cert\") pod \"19e20106-ec71-4202-aabd-914e728ac6de\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " Jun 02 19:30:12.813627 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813591 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-serving-cert\") pod \"19e20106-ec71-4202-aabd-914e728ac6de\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " Jun 02 19:30:12.813627 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813615 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-trusted-ca-bundle\") pod \"19e20106-ec71-4202-aabd-914e728ac6de\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " Jun 02 19:30:12.813692 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813649 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-oauth-config\") pod \"19e20106-ec71-4202-aabd-914e728ac6de\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " Jun 02 19:30:12.813692 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813684 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-console-config\") pod \"19e20106-ec71-4202-aabd-914e728ac6de\" (UID: \"19e20106-ec71-4202-aabd-914e728ac6de\") " Jun 02 19:30:12.813928 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813737 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-service-ca" (OuterVolumeSpecName: "service-ca") pod "19e20106-ec71-4202-aabd-914e728ac6de" (UID: "19e20106-ec71-4202-aabd-914e728ac6de"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:30:12.813928 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.813895 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "19e20106-ec71-4202-aabd-914e728ac6de" (UID: "19e20106-ec71-4202-aabd-914e728ac6de"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:30:12.814139 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.814071 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-service-ca\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:12.814139 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.814073 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "19e20106-ec71-4202-aabd-914e728ac6de" (UID: "19e20106-ec71-4202-aabd-914e728ac6de"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:30:12.814139 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.814085 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-oauth-serving-cert\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:12.814297 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.814159 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-console-config" (OuterVolumeSpecName: "console-config") pod "19e20106-ec71-4202-aabd-914e728ac6de" (UID: "19e20106-ec71-4202-aabd-914e728ac6de"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:30:12.815867 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.815842 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "19e20106-ec71-4202-aabd-914e728ac6de" (UID: "19e20106-ec71-4202-aabd-914e728ac6de"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:30:12.815980 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.815944 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19e20106-ec71-4202-aabd-914e728ac6de-kube-api-access-fxkgq" (OuterVolumeSpecName: "kube-api-access-fxkgq") pod "19e20106-ec71-4202-aabd-914e728ac6de" (UID: "19e20106-ec71-4202-aabd-914e728ac6de"). InnerVolumeSpecName "kube-api-access-fxkgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:30:12.816119 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.816090 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "19e20106-ec71-4202-aabd-914e728ac6de" (UID: "19e20106-ec71-4202-aabd-914e728ac6de"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:30:12.914989 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.914942 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-serving-cert\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:12.914989 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.914978 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-trusted-ca-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:12.914989 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.914997 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19e20106-ec71-4202-aabd-914e728ac6de-console-oauth-config\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:12.915241 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.915011 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19e20106-ec71-4202-aabd-914e728ac6de-console-config\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:12.915241 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:12.915025 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxkgq\" (UniqueName: \"kubernetes.io/projected/19e20106-ec71-4202-aabd-914e728ac6de-kube-api-access-fxkgq\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:13.439362 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.439333 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-795c9c8864-9trx6_19e20106-ec71-4202-aabd-914e728ac6de/console/0.log" Jun 02 19:30:13.439820 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.439373 2552 generic.go:358] "Generic (PLEG): container finished" podID="19e20106-ec71-4202-aabd-914e728ac6de" containerID="7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1" exitCode=2 Jun 02 19:30:13.439820 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.439409 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795c9c8864-9trx6" event={"ID":"19e20106-ec71-4202-aabd-914e728ac6de","Type":"ContainerDied","Data":"7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1"} Jun 02 19:30:13.439820 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.439434 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-795c9c8864-9trx6" Jun 02 19:30:13.439820 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.439450 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-795c9c8864-9trx6" event={"ID":"19e20106-ec71-4202-aabd-914e728ac6de","Type":"ContainerDied","Data":"d5793d83979feec3c764c7558572b47f1c5ca6490fe2716b8753769d0a063ddb"} Jun 02 19:30:13.439820 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.439468 2552 scope.go:117] "RemoveContainer" containerID="7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1" Jun 02 19:30:13.449375 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.449355 2552 scope.go:117] "RemoveContainer" containerID="7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1" Jun 02 19:30:13.449679 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:30:13.449660 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1\": container with ID starting with 7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1 not found: ID does not exist" containerID="7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1" Jun 02 19:30:13.449728 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.449688 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1"} err="failed to get container status \"7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1\": rpc error: code = NotFound desc = could not find container \"7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1\": container with ID starting with 7a3735ca7dc5a00c2b63514e0005fed05cce101c389440c9ca5419983494e1a1 not found: ID does not exist" Jun 02 19:30:13.466403 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.466371 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-795c9c8864-9trx6"] Jun 02 19:30:13.469057 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:13.469031 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-795c9c8864-9trx6"] Jun 02 19:30:14.394731 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:14.394694 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19e20106-ec71-4202-aabd-914e728ac6de" path="/var/lib/kubelet/pods/19e20106-ec71-4202-aabd-914e728ac6de/volumes" Jun 02 19:30:20.066360 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.066322 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4"] Jun 02 19:30:20.066892 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.066794 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19e20106-ec71-4202-aabd-914e728ac6de" containerName="console" Jun 02 19:30:20.066892 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.066808 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="19e20106-ec71-4202-aabd-914e728ac6de" containerName="console" Jun 02 19:30:20.066892 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.066878 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="19e20106-ec71-4202-aabd-914e728ac6de" containerName="console" Jun 02 19:30:20.072003 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.071981 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.076752 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.076531 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dv7bn\"" Jun 02 19:30:20.076948 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.076920 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 19:30:20.077088 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.076999 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 19:30:20.079981 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.079947 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4"] Jun 02 19:30:20.086840 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.086814 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.086949 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.086867 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.086949 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.086907 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtzkq\" (UniqueName: \"kubernetes.io/projected/33b106b5-189e-4cdd-a470-24a7aa180809-kube-api-access-qtzkq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.187575 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.187531 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.187777 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.187609 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.187777 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.187673 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qtzkq\" (UniqueName: \"kubernetes.io/projected/33b106b5-189e-4cdd-a470-24a7aa180809-kube-api-access-qtzkq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.187916 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.187892 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.187972 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.187939 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.196690 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.196665 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtzkq\" (UniqueName: \"kubernetes.io/projected/33b106b5-189e-4cdd-a470-24a7aa180809-kube-api-access-qtzkq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.383483 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.383391 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:20.523328 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.523297 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4"] Jun 02 19:30:20.525609 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:30:20.525583 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b106b5_189e_4cdd_a470_24a7aa180809.slice/crio-22492ad9f8057e50df56e43e9d52220f5706bdd4b4670bc02c8a2c5ddb457ec2 WatchSource:0}: Error finding container 22492ad9f8057e50df56e43e9d52220f5706bdd4b4670bc02c8a2c5ddb457ec2: Status 404 returned error can't find the container with id 22492ad9f8057e50df56e43e9d52220f5706bdd4b4670bc02c8a2c5ddb457ec2 Jun 02 19:30:20.849650 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.849599 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b"] Jun 02 19:30:20.853563 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.853539 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:20.865374 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.865337 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b"] Jun 02 19:30:20.892079 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.892002 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt2xm\" (UniqueName: \"kubernetes.io/projected/184ffb75-84b8-4a0d-8905-f7894e460c8d-kube-api-access-lt2xm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:20.892079 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.892061 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:20.892247 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.892117 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:20.993042 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.993001 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lt2xm\" (UniqueName: \"kubernetes.io/projected/184ffb75-84b8-4a0d-8905-f7894e460c8d-kube-api-access-lt2xm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:20.993042 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.993048 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:20.993311 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.993088 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:20.993471 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.993449 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:20.993558 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:20.993470 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:21.001587 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.001560 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt2xm\" (UniqueName: \"kubernetes.io/projected/184ffb75-84b8-4a0d-8905-f7894e460c8d-kube-api-access-lt2xm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:21.168227 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.168116 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:21.308301 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.308275 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b"] Jun 02 19:30:21.310155 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:30:21.310126 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184ffb75_84b8_4a0d_8905_f7894e460c8d.slice/crio-a21356415308e7cfe6c16dbba50898169600772bcefcd04f3828d2f4fdb4d230 WatchSource:0}: Error finding container a21356415308e7cfe6c16dbba50898169600772bcefcd04f3828d2f4fdb4d230: Status 404 returned error can't find the container with id a21356415308e7cfe6c16dbba50898169600772bcefcd04f3828d2f4fdb4d230 Jun 02 19:30:21.474719 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.474685 2552 generic.go:358] "Generic (PLEG): container finished" podID="33b106b5-189e-4cdd-a470-24a7aa180809" containerID="cce3b63574a3bae0b46bca663f0d0bf265dfb4ff9b50944080d91fd3d0a7b12a" exitCode=0 Jun 02 19:30:21.474896 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.474769 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" event={"ID":"33b106b5-189e-4cdd-a470-24a7aa180809","Type":"ContainerDied","Data":"cce3b63574a3bae0b46bca663f0d0bf265dfb4ff9b50944080d91fd3d0a7b12a"} Jun 02 19:30:21.474896 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.474803 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" event={"ID":"33b106b5-189e-4cdd-a470-24a7aa180809","Type":"ContainerStarted","Data":"22492ad9f8057e50df56e43e9d52220f5706bdd4b4670bc02c8a2c5ddb457ec2"} Jun 02 19:30:21.476306 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.476283 2552 generic.go:358] "Generic (PLEG): container finished" podID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerID="0617ad8567d4fdb6595a2447885baa90223a79686a1d87fc9af3636623d4d0f8" exitCode=0 Jun 02 19:30:21.476381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.476332 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" event={"ID":"184ffb75-84b8-4a0d-8905-f7894e460c8d","Type":"ContainerDied","Data":"0617ad8567d4fdb6595a2447885baa90223a79686a1d87fc9af3636623d4d0f8"} Jun 02 19:30:21.476381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:21.476354 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" event={"ID":"184ffb75-84b8-4a0d-8905-f7894e460c8d","Type":"ContainerStarted","Data":"a21356415308e7cfe6c16dbba50898169600772bcefcd04f3828d2f4fdb4d230"} Jun 02 19:30:22.481885 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:22.481849 2552 generic.go:358] "Generic (PLEG): container finished" podID="33b106b5-189e-4cdd-a470-24a7aa180809" containerID="cb82132f9ccc6ecf970fad1d1fad08b1dd9a29baa10bb20935a50b66eaaf4fd9" exitCode=0 Jun 02 19:30:22.482321 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:22.481929 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" event={"ID":"33b106b5-189e-4cdd-a470-24a7aa180809","Type":"ContainerDied","Data":"cb82132f9ccc6ecf970fad1d1fad08b1dd9a29baa10bb20935a50b66eaaf4fd9"} Jun 02 19:30:22.483759 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:22.483737 2552 generic.go:358] "Generic (PLEG): container finished" podID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerID="55b5db7ad9b7543270ac4b5662f20f9a43544ea029b530c60b812928c30519e7" exitCode=0 Jun 02 19:30:22.483849 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:22.483833 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" event={"ID":"184ffb75-84b8-4a0d-8905-f7894e460c8d","Type":"ContainerDied","Data":"55b5db7ad9b7543270ac4b5662f20f9a43544ea029b530c60b812928c30519e7"} Jun 02 19:30:23.489512 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:23.489455 2552 generic.go:358] "Generic (PLEG): container finished" podID="33b106b5-189e-4cdd-a470-24a7aa180809" containerID="5fddeff4425c72e266c5fdb4c54bd6545bffc04a5ad2be5aaa9cf15595d6785d" exitCode=0 Jun 02 19:30:23.489917 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:23.489509 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" event={"ID":"33b106b5-189e-4cdd-a470-24a7aa180809","Type":"ContainerDied","Data":"5fddeff4425c72e266c5fdb4c54bd6545bffc04a5ad2be5aaa9cf15595d6785d"} Jun 02 19:30:23.491429 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:23.491401 2552 generic.go:358] "Generic (PLEG): container finished" podID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerID="d24bc4f16ced59142f35a72c886c25e93ac97d48b9e0e264c109a4072eaaf364" exitCode=0 Jun 02 19:30:23.491516 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:23.491440 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" event={"ID":"184ffb75-84b8-4a0d-8905-f7894e460c8d","Type":"ContainerDied","Data":"d24bc4f16ced59142f35a72c886c25e93ac97d48b9e0e264c109a4072eaaf364"} Jun 02 19:30:24.657626 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.657598 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:24.660839 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.660819 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:24.723762 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.723726 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtzkq\" (UniqueName: \"kubernetes.io/projected/33b106b5-189e-4cdd-a470-24a7aa180809-kube-api-access-qtzkq\") pod \"33b106b5-189e-4cdd-a470-24a7aa180809\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " Jun 02 19:30:24.723762 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.723769 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-util\") pod \"184ffb75-84b8-4a0d-8905-f7894e460c8d\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " Jun 02 19:30:24.723982 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.723802 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt2xm\" (UniqueName: \"kubernetes.io/projected/184ffb75-84b8-4a0d-8905-f7894e460c8d-kube-api-access-lt2xm\") pod \"184ffb75-84b8-4a0d-8905-f7894e460c8d\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " Jun 02 19:30:24.723982 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.723828 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-util\") pod \"33b106b5-189e-4cdd-a470-24a7aa180809\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " Jun 02 19:30:24.723982 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.723889 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-bundle\") pod \"184ffb75-84b8-4a0d-8905-f7894e460c8d\" (UID: \"184ffb75-84b8-4a0d-8905-f7894e460c8d\") " Jun 02 19:30:24.723982 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.723934 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-bundle\") pod \"33b106b5-189e-4cdd-a470-24a7aa180809\" (UID: \"33b106b5-189e-4cdd-a470-24a7aa180809\") " Jun 02 19:30:24.724404 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.724380 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-bundle" (OuterVolumeSpecName: "bundle") pod "184ffb75-84b8-4a0d-8905-f7894e460c8d" (UID: "184ffb75-84b8-4a0d-8905-f7894e460c8d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:30:24.724715 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.724685 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-bundle" (OuterVolumeSpecName: "bundle") pod "33b106b5-189e-4cdd-a470-24a7aa180809" (UID: "33b106b5-189e-4cdd-a470-24a7aa180809"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:30:24.726181 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.726151 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b106b5-189e-4cdd-a470-24a7aa180809-kube-api-access-qtzkq" (OuterVolumeSpecName: "kube-api-access-qtzkq") pod "33b106b5-189e-4cdd-a470-24a7aa180809" (UID: "33b106b5-189e-4cdd-a470-24a7aa180809"). InnerVolumeSpecName "kube-api-access-qtzkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:30:24.726181 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.726171 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184ffb75-84b8-4a0d-8905-f7894e460c8d-kube-api-access-lt2xm" (OuterVolumeSpecName: "kube-api-access-lt2xm") pod "184ffb75-84b8-4a0d-8905-f7894e460c8d" (UID: "184ffb75-84b8-4a0d-8905-f7894e460c8d"). InnerVolumeSpecName "kube-api-access-lt2xm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:30:24.732951 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.732895 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-util" (OuterVolumeSpecName: "util") pod "33b106b5-189e-4cdd-a470-24a7aa180809" (UID: "33b106b5-189e-4cdd-a470-24a7aa180809"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:30:24.733040 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.733004 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-util" (OuterVolumeSpecName: "util") pod "184ffb75-84b8-4a0d-8905-f7894e460c8d" (UID: "184ffb75-84b8-4a0d-8905-f7894e460c8d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:30:24.825730 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.825696 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt2xm\" (UniqueName: \"kubernetes.io/projected/184ffb75-84b8-4a0d-8905-f7894e460c8d-kube-api-access-lt2xm\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:24.825730 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.825727 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-util\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:24.825730 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.825738 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:24.825967 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.825746 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33b106b5-189e-4cdd-a470-24a7aa180809-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:24.825967 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.825756 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtzkq\" (UniqueName: \"kubernetes.io/projected/33b106b5-189e-4cdd-a470-24a7aa180809-kube-api-access-qtzkq\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:24.825967 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:24.825765 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/184ffb75-84b8-4a0d-8905-f7894e460c8d-util\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:30:25.503561 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:25.503515 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" event={"ID":"33b106b5-189e-4cdd-a470-24a7aa180809","Type":"ContainerDied","Data":"22492ad9f8057e50df56e43e9d52220f5706bdd4b4670bc02c8a2c5ddb457ec2"} Jun 02 19:30:25.503561 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:25.503565 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22492ad9f8057e50df56e43e9d52220f5706bdd4b4670bc02c8a2c5ddb457ec2" Jun 02 19:30:25.503806 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:25.503536 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4" Jun 02 19:30:25.505302 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:25.505261 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" event={"ID":"184ffb75-84b8-4a0d-8905-f7894e460c8d","Type":"ContainerDied","Data":"a21356415308e7cfe6c16dbba50898169600772bcefcd04f3828d2f4fdb4d230"} Jun 02 19:30:25.505302 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:25.505302 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a21356415308e7cfe6c16dbba50898169600772bcefcd04f3828d2f4fdb4d230" Jun 02 19:30:25.505455 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:25.505322 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b" Jun 02 19:30:40.365197 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365157 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk"] Jun 02 19:30:40.365788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365726 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b106b5-189e-4cdd-a470-24a7aa180809" containerName="util" Jun 02 19:30:40.365788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365744 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b106b5-189e-4cdd-a470-24a7aa180809" containerName="util" Jun 02 19:30:40.365788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365756 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerName="util" Jun 02 19:30:40.365788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365763 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerName="util" Jun 02 19:30:40.365788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365774 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b106b5-189e-4cdd-a470-24a7aa180809" containerName="extract" Jun 02 19:30:40.365788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365779 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b106b5-189e-4cdd-a470-24a7aa180809" containerName="extract" Jun 02 19:30:40.365788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365788 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerName="extract" Jun 02 19:30:40.365788 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365793 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerName="extract" Jun 02 19:30:40.366037 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365808 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b106b5-189e-4cdd-a470-24a7aa180809" containerName="pull" Jun 02 19:30:40.366037 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365813 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b106b5-189e-4cdd-a470-24a7aa180809" containerName="pull" Jun 02 19:30:40.366037 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365819 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerName="pull" Jun 02 19:30:40.366037 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365824 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerName="pull" Jun 02 19:30:40.366037 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365884 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="184ffb75-84b8-4a0d-8905-f7894e460c8d" containerName="extract" Jun 02 19:30:40.366037 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.365893 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="33b106b5-189e-4cdd-a470-24a7aa180809" containerName="extract" Jun 02 19:30:40.369271 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.369253 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" Jun 02 19:30:40.372240 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.372208 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-llnwz\"" Jun 02 19:30:40.372728 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.372589 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 19:30:40.373319 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.373292 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 02 19:30:40.373814 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.373795 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 19:30:40.382189 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.382157 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk"] Jun 02 19:30:40.479234 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.479192 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqcqq\" (UniqueName: \"kubernetes.io/projected/ac79c3e7-c694-418c-b958-2059ec5f5e3d-kube-api-access-pqcqq\") pod \"dns-operator-controller-manager-648d5c98bc-hz4qk\" (UID: \"ac79c3e7-c694-418c-b958-2059ec5f5e3d\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" Jun 02 19:30:40.580669 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.580629 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pqcqq\" (UniqueName: \"kubernetes.io/projected/ac79c3e7-c694-418c-b958-2059ec5f5e3d-kube-api-access-pqcqq\") pod \"dns-operator-controller-manager-648d5c98bc-hz4qk\" (UID: \"ac79c3e7-c694-418c-b958-2059ec5f5e3d\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" Jun 02 19:30:40.601845 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.601803 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqcqq\" (UniqueName: \"kubernetes.io/projected/ac79c3e7-c694-418c-b958-2059ec5f5e3d-kube-api-access-pqcqq\") pod \"dns-operator-controller-manager-648d5c98bc-hz4qk\" (UID: \"ac79c3e7-c694-418c-b958-2059ec5f5e3d\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" Jun 02 19:30:40.690062 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.689962 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" Jun 02 19:30:40.833927 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:40.833896 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk"] Jun 02 19:30:40.835585 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:30:40.835558 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac79c3e7_c694_418c_b958_2059ec5f5e3d.slice/crio-7968a620e06d430c64d540c0fb14aea628cd8628c7c5057d3427f22b0111afac WatchSource:0}: Error finding container 7968a620e06d430c64d540c0fb14aea628cd8628c7c5057d3427f22b0111afac: Status 404 returned error can't find the container with id 7968a620e06d430c64d540c0fb14aea628cd8628c7c5057d3427f22b0111afac Jun 02 19:30:41.569010 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:41.568967 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" event={"ID":"ac79c3e7-c694-418c-b958-2059ec5f5e3d","Type":"ContainerStarted","Data":"7968a620e06d430c64d540c0fb14aea628cd8628c7c5057d3427f22b0111afac"} Jun 02 19:30:43.577723 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:43.577687 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" event={"ID":"ac79c3e7-c694-418c-b958-2059ec5f5e3d","Type":"ContainerStarted","Data":"e737de8617fa31433d6f4d58a7a3fc52f72880f7670efa8b3c009c237bd07140"} Jun 02 19:30:43.578128 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:43.577864 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" Jun 02 19:30:43.604476 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:43.604416 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" podStartSLOduration=1.177806622 podStartE2EDuration="3.604400816s" podCreationTimestamp="2026-06-02 19:30:40 +0000 UTC" firstStartedPulling="2026-06-02 19:30:40.837688552 +0000 UTC m=+570.927556762" lastFinishedPulling="2026-06-02 19:30:43.264282732 +0000 UTC m=+573.354150956" observedRunningTime="2026-06-02 19:30:43.600932715 +0000 UTC m=+573.690800966" watchObservedRunningTime="2026-06-02 19:30:43.604400816 +0000 UTC m=+573.694269122" Jun 02 19:30:44.451475 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.451436 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm"] Jun 02 19:30:44.455175 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.455147 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:44.461416 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.461391 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dg7pj\"" Jun 02 19:30:44.471175 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.471145 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm"] Jun 02 19:30:44.520814 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.520765 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57d1c000-f01e-4221-819c-8441a150cc07-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" (UID: \"57d1c000-f01e-4221-819c-8441a150cc07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:44.520814 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.520811 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87fzr\" (UniqueName: \"kubernetes.io/projected/57d1c000-f01e-4221-819c-8441a150cc07-kube-api-access-87fzr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" (UID: \"57d1c000-f01e-4221-819c-8441a150cc07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:44.621589 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.621547 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57d1c000-f01e-4221-819c-8441a150cc07-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" (UID: \"57d1c000-f01e-4221-819c-8441a150cc07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:44.621589 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.621587 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-87fzr\" (UniqueName: \"kubernetes.io/projected/57d1c000-f01e-4221-819c-8441a150cc07-kube-api-access-87fzr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" (UID: \"57d1c000-f01e-4221-819c-8441a150cc07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:44.622031 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.621954 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57d1c000-f01e-4221-819c-8441a150cc07-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" (UID: \"57d1c000-f01e-4221-819c-8441a150cc07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:44.632807 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.632770 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87fzr\" (UniqueName: \"kubernetes.io/projected/57d1c000-f01e-4221-819c-8441a150cc07-kube-api-access-87fzr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" (UID: \"57d1c000-f01e-4221-819c-8441a150cc07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:44.765530 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.765398 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:44.907184 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:44.907139 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm"] Jun 02 19:30:44.911751 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:30:44.911718 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d1c000_f01e_4221_819c_8441a150cc07.slice/crio-0a9cca993f32412c4d30c00dc8de9ce743d0a2626f68781b395c3ed3813abd7f WatchSource:0}: Error finding container 0a9cca993f32412c4d30c00dc8de9ce743d0a2626f68781b395c3ed3813abd7f: Status 404 returned error can't find the container with id 0a9cca993f32412c4d30c00dc8de9ce743d0a2626f68781b395c3ed3813abd7f Jun 02 19:30:45.587795 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:45.587754 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" event={"ID":"57d1c000-f01e-4221-819c-8441a150cc07","Type":"ContainerStarted","Data":"0a9cca993f32412c4d30c00dc8de9ce743d0a2626f68781b395c3ed3813abd7f"} Jun 02 19:30:50.617864 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:50.617825 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" event={"ID":"57d1c000-f01e-4221-819c-8441a150cc07","Type":"ContainerStarted","Data":"9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00"} Jun 02 19:30:50.618303 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:50.617982 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:30:50.640149 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:50.640091 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" podStartSLOduration=1.718990824 podStartE2EDuration="6.640075957s" podCreationTimestamp="2026-06-02 19:30:44 +0000 UTC" firstStartedPulling="2026-06-02 19:30:44.914378562 +0000 UTC m=+575.004246772" lastFinishedPulling="2026-06-02 19:30:49.83546368 +0000 UTC m=+579.925331905" observedRunningTime="2026-06-02 19:30:50.639600224 +0000 UTC m=+580.729468460" watchObservedRunningTime="2026-06-02 19:30:50.640075957 +0000 UTC m=+580.729944189" Jun 02 19:30:54.585046 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:30:54.585006 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-hz4qk" Jun 02 19:31:01.625387 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:01.625352 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:31:02.905236 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:02.905189 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs"] Jun 02 19:31:02.915761 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:02.915729 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:02.923687 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:02.923430 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs"] Jun 02 19:31:03.000398 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.000365 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7de0e1e-eda4-43d0-a9a3-123403baf57c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" (UID: \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:03.000635 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.000435 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxvg7\" (UniqueName: \"kubernetes.io/projected/f7de0e1e-eda4-43d0-a9a3-123403baf57c-kube-api-access-wxvg7\") pod \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" (UID: \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:03.101902 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.101862 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7de0e1e-eda4-43d0-a9a3-123403baf57c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" (UID: \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:03.102084 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.101940 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxvg7\" (UniqueName: \"kubernetes.io/projected/f7de0e1e-eda4-43d0-a9a3-123403baf57c-kube-api-access-wxvg7\") pod \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" (UID: \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:03.102266 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.102247 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7de0e1e-eda4-43d0-a9a3-123403baf57c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" (UID: \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:03.114469 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.114439 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxvg7\" (UniqueName: \"kubernetes.io/projected/f7de0e1e-eda4-43d0-a9a3-123403baf57c-kube-api-access-wxvg7\") pod \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" (UID: \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:03.228214 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.228107 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:03.343873 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.343839 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm"] Jun 02 19:31:03.344132 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.344087 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" podUID="57d1c000-f01e-4221-819c-8441a150cc07" containerName="manager" containerID="cri-o://9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00" gracePeriod=2 Jun 02 19:31:03.354017 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.353983 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm"] Jun 02 19:31:03.377314 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.377207 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz"] Jun 02 19:31:03.377912 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.377881 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57d1c000-f01e-4221-819c-8441a150cc07" containerName="manager" Jun 02 19:31:03.377912 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.377906 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="57d1c000-f01e-4221-819c-8441a150cc07" containerName="manager" Jun 02 19:31:03.378067 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.378019 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="57d1c000-f01e-4221-819c-8441a150cc07" containerName="manager" Jun 02 19:31:03.381745 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.381724 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs"] Jun 02 19:31:03.381856 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.381845 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:03.391339 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.391311 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz"] Jun 02 19:31:03.401297 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.401266 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs"] Jun 02 19:31:03.403323 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.403287 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs"] Jun 02 19:31:03.404797 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.404770 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e7218be-fbc0-4295-bf48-331d1de0fe9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-d7scz\" (UID: \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:03.404910 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.404813 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhz6m\" (UniqueName: \"kubernetes.io/projected/9e7218be-fbc0-4295-bf48-331d1de0fe9d-kube-api-access-rhz6m\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-d7scz\" (UID: \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:03.411744 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:31:03.411711 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7de0e1e_eda4_43d0_a9a3_123403baf57c.slice/crio-5bb89433aacc522715cfa891ba46f45d87e9daac837963a78d9e67ad119c9ff9 WatchSource:0}: Error finding container 5bb89433aacc522715cfa891ba46f45d87e9daac837963a78d9e67ad119c9ff9: Status 404 returned error can't find the container with id 5bb89433aacc522715cfa891ba46f45d87e9daac837963a78d9e67ad119c9ff9 Jun 02 19:31:03.431334 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.431280 2552 status_manager.go:895] "Failed to get status for pod" podUID="57d1c000-f01e-4221-819c-8441a150cc07" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.441854 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.441803 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn"] Jun 02 19:31:03.446201 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.446169 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" Jun 02 19:31:03.449881 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.449834 2552 status_manager.go:895] "Failed to get status for pod" podUID="57d1c000-f01e-4221-819c-8441a150cc07" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.450615 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.450538 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bv8vp\"" Jun 02 19:31:03.457720 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.457693 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn"] Jun 02 19:31:03.505808 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.505674 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rhz6m\" (UniqueName: \"kubernetes.io/projected/9e7218be-fbc0-4295-bf48-331d1de0fe9d-kube-api-access-rhz6m\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-d7scz\" (UID: \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:03.505808 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.505732 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk6gz\" (UniqueName: \"kubernetes.io/projected/3eb2d116-9ca7-47b1-a788-7a02ff1c7a68-kube-api-access-lk6gz\") pod \"limitador-operator-controller-manager-85c4996f8c-q7cxn\" (UID: \"3eb2d116-9ca7-47b1-a788-7a02ff1c7a68\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" Jun 02 19:31:03.506348 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.506321 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e7218be-fbc0-4295-bf48-331d1de0fe9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-d7scz\" (UID: \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:03.506798 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.506771 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e7218be-fbc0-4295-bf48-331d1de0fe9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-d7scz\" (UID: \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:03.521213 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.521159 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhz6m\" (UniqueName: \"kubernetes.io/projected/9e7218be-fbc0-4295-bf48-331d1de0fe9d-kube-api-access-rhz6m\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-d7scz\" (UID: \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:03.578436 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.578411 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:31:03.581234 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.581198 2552 status_manager.go:895] "Failed to get status for pod" podUID="57d1c000-f01e-4221-819c-8441a150cc07" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.607793 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.607755 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87fzr\" (UniqueName: \"kubernetes.io/projected/57d1c000-f01e-4221-819c-8441a150cc07-kube-api-access-87fzr\") pod \"57d1c000-f01e-4221-819c-8441a150cc07\" (UID: \"57d1c000-f01e-4221-819c-8441a150cc07\") " Jun 02 19:31:03.607974 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.607808 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57d1c000-f01e-4221-819c-8441a150cc07-extensions-socket-volume\") pod \"57d1c000-f01e-4221-819c-8441a150cc07\" (UID: \"57d1c000-f01e-4221-819c-8441a150cc07\") " Jun 02 19:31:03.607974 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.607929 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lk6gz\" (UniqueName: \"kubernetes.io/projected/3eb2d116-9ca7-47b1-a788-7a02ff1c7a68-kube-api-access-lk6gz\") pod \"limitador-operator-controller-manager-85c4996f8c-q7cxn\" (UID: \"3eb2d116-9ca7-47b1-a788-7a02ff1c7a68\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" Jun 02 19:31:03.608410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.608378 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57d1c000-f01e-4221-819c-8441a150cc07-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "57d1c000-f01e-4221-819c-8441a150cc07" (UID: "57d1c000-f01e-4221-819c-8441a150cc07"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:31:03.610098 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.610073 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57d1c000-f01e-4221-819c-8441a150cc07-kube-api-access-87fzr" (OuterVolumeSpecName: "kube-api-access-87fzr") pod "57d1c000-f01e-4221-819c-8441a150cc07" (UID: "57d1c000-f01e-4221-819c-8441a150cc07"). InnerVolumeSpecName "kube-api-access-87fzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:31:03.641897 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.641861 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk6gz\" (UniqueName: \"kubernetes.io/projected/3eb2d116-9ca7-47b1-a788-7a02ff1c7a68-kube-api-access-lk6gz\") pod \"limitador-operator-controller-manager-85c4996f8c-q7cxn\" (UID: \"3eb2d116-9ca7-47b1-a788-7a02ff1c7a68\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" Jun 02 19:31:03.671141 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.671101 2552 generic.go:358] "Generic (PLEG): container finished" podID="57d1c000-f01e-4221-819c-8441a150cc07" containerID="9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00" exitCode=0 Jun 02 19:31:03.671347 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.671152 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" Jun 02 19:31:03.671347 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.671214 2552 scope.go:117] "RemoveContainer" containerID="9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00" Jun 02 19:31:03.673007 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.672961 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" containerName="manager" containerID="cri-o://1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817" gracePeriod=2 Jun 02 19:31:03.674328 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.674299 2552 status_manager.go:895] "Failed to get status for pod" podUID="57d1c000-f01e-4221-819c-8441a150cc07" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.677123 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.677093 2552 status_manager.go:895] "Failed to get status for pod" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" err="pods \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.679872 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.679834 2552 status_manager.go:895] "Failed to get status for pod" podUID="57d1c000-f01e-4221-819c-8441a150cc07" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.680910 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.680894 2552 scope.go:117] "RemoveContainer" containerID="9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00" Jun 02 19:31:03.681198 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:31:03.681180 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00\": container with ID starting with 9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00 not found: ID does not exist" containerID="9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00" Jun 02 19:31:03.681250 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.681208 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00"} err="failed to get container status \"9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00\": rpc error: code = NotFound desc = could not find container \"9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00\": container with ID starting with 9899d1b3c404acfb3c02c017ae79db6bec13fe615a65eefe8e1b2cd4b8e47e00 not found: ID does not exist" Jun 02 19:31:03.685749 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.685722 2552 status_manager.go:895] "Failed to get status for pod" podUID="57d1c000-f01e-4221-819c-8441a150cc07" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.688317 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.688296 2552 status_manager.go:895] "Failed to get status for pod" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" err="pods \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.708842 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.708811 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57d1c000-f01e-4221-819c-8441a150cc07-extensions-socket-volume\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:31:03.708842 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.708841 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87fzr\" (UniqueName: \"kubernetes.io/projected/57d1c000-f01e-4221-819c-8441a150cc07-kube-api-access-87fzr\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:31:03.726556 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.726526 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:03.761568 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.761535 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" Jun 02 19:31:03.885233 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.885203 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz"] Jun 02 19:31:03.889124 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:31:03.889065 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e7218be_fbc0_4295_bf48_331d1de0fe9d.slice/crio-e7484c1fbdbdaa5922ad7ee3bd945ac09f99cc89b81437faaea0fab5d961cc55 WatchSource:0}: Error finding container e7484c1fbdbdaa5922ad7ee3bd945ac09f99cc89b81437faaea0fab5d961cc55: Status 404 returned error can't find the container with id e7484c1fbdbdaa5922ad7ee3bd945ac09f99cc89b81437faaea0fab5d961cc55 Jun 02 19:31:03.929885 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.929862 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn"] Jun 02 19:31:03.941924 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:31:03.941892 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb2d116_9ca7_47b1_a788_7a02ff1c7a68.slice/crio-f4bf1fb03e08589681dc7a6c94ac846568c147e9438d95267c56ae9694aae1d0 WatchSource:0}: Error finding container f4bf1fb03e08589681dc7a6c94ac846568c147e9438d95267c56ae9694aae1d0: Status 404 returned error can't find the container with id f4bf1fb03e08589681dc7a6c94ac846568c147e9438d95267c56ae9694aae1d0 Jun 02 19:31:03.951393 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.951371 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:03.953802 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.953774 2552 status_manager.go:895] "Failed to get status for pod" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" err="pods \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:03.955787 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:03.955764 2552 status_manager.go:895] "Failed to get status for pod" podUID="57d1c000-f01e-4221-819c-8441a150cc07" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-dfpmm" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-dfpmm\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:04.012002 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.011896 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxvg7\" (UniqueName: \"kubernetes.io/projected/f7de0e1e-eda4-43d0-a9a3-123403baf57c-kube-api-access-wxvg7\") pod \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\" (UID: \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\") " Jun 02 19:31:04.012002 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.011996 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7de0e1e-eda4-43d0-a9a3-123403baf57c-extensions-socket-volume\") pod \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\" (UID: \"f7de0e1e-eda4-43d0-a9a3-123403baf57c\") " Jun 02 19:31:04.012285 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.012252 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7de0e1e-eda4-43d0-a9a3-123403baf57c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f7de0e1e-eda4-43d0-a9a3-123403baf57c" (UID: "f7de0e1e-eda4-43d0-a9a3-123403baf57c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:31:04.014047 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.014028 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7de0e1e-eda4-43d0-a9a3-123403baf57c-kube-api-access-wxvg7" (OuterVolumeSpecName: "kube-api-access-wxvg7") pod "f7de0e1e-eda4-43d0-a9a3-123403baf57c" (UID: "f7de0e1e-eda4-43d0-a9a3-123403baf57c"). InnerVolumeSpecName "kube-api-access-wxvg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:31:04.112936 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.112898 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7de0e1e-eda4-43d0-a9a3-123403baf57c-extensions-socket-volume\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:31:04.112936 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.112928 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxvg7\" (UniqueName: \"kubernetes.io/projected/f7de0e1e-eda4-43d0-a9a3-123403baf57c-kube-api-access-wxvg7\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:31:04.395538 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.395484 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57d1c000-f01e-4221-819c-8441a150cc07" path="/var/lib/kubelet/pods/57d1c000-f01e-4221-819c-8441a150cc07/volumes" Jun 02 19:31:04.396005 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.395984 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" path="/var/lib/kubelet/pods/f7de0e1e-eda4-43d0-a9a3-123403baf57c/volumes" Jun 02 19:31:04.681724 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.681630 2552 generic.go:358] "Generic (PLEG): container finished" podID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" containerID="1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817" exitCode=2 Jun 02 19:31:04.681724 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.681694 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" Jun 02 19:31:04.681947 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.681745 2552 scope.go:117] "RemoveContainer" containerID="1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817" Jun 02 19:31:04.684291 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.683973 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" event={"ID":"3eb2d116-9ca7-47b1-a788-7a02ff1c7a68","Type":"ContainerStarted","Data":"f4bf1fb03e08589681dc7a6c94ac846568c147e9438d95267c56ae9694aae1d0"} Jun 02 19:31:04.686405 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.686376 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" event={"ID":"9e7218be-fbc0-4295-bf48-331d1de0fe9d","Type":"ContainerStarted","Data":"7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357"} Jun 02 19:31:04.686405 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.686413 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" event={"ID":"9e7218be-fbc0-4295-bf48-331d1de0fe9d","Type":"ContainerStarted","Data":"e7484c1fbdbdaa5922ad7ee3bd945ac09f99cc89b81437faaea0fab5d961cc55"} Jun 02 19:31:04.686619 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.686535 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:04.687527 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.687468 2552 status_manager.go:895] "Failed to get status for pod" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" err="pods \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:04.689995 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.689971 2552 status_manager.go:895] "Failed to get status for pod" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" err="pods \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:04.700359 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.700330 2552 scope.go:117] "RemoveContainer" containerID="1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817" Jun 02 19:31:04.701157 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:31:04.701087 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817\": container with ID starting with 1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817 not found: ID does not exist" containerID="1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817" Jun 02 19:31:04.701157 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.701132 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817"} err="failed to get container status \"1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817\": rpc error: code = NotFound desc = could not find container \"1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817\": container with ID starting with 1b233efc64a82b7cec53f6b08906211534041fe7f8e4cc13a538df45190e6817 not found: ID does not exist" Jun 02 19:31:04.743058 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:04.742993 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" podStartSLOduration=1.742975368 podStartE2EDuration="1.742975368s" podCreationTimestamp="2026-06-02 19:31:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:31:04.737523902 +0000 UTC m=+594.827392133" watchObservedRunningTime="2026-06-02 19:31:04.742975368 +0000 UTC m=+594.832843602" Jun 02 19:31:05.697145 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:05.697096 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" event={"ID":"3eb2d116-9ca7-47b1-a788-7a02ff1c7a68","Type":"ContainerStarted","Data":"d9f39e0070630930424a34e336a1f412c05affdef23924dc39340b2992ca13aa"} Jun 02 19:31:05.697596 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:05.697168 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" Jun 02 19:31:05.699524 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:05.699481 2552 status_manager.go:895] "Failed to get status for pod" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-kbbjs" err="pods \"kuadrant-operator-controller-manager-84b657d985-kbbjs\" is forbidden: User \"system:node:ip-10-0-138-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-231.ec2.internal' and this object" Jun 02 19:31:05.717899 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:05.717852 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" podStartSLOduration=1.471080827 podStartE2EDuration="2.717837698s" podCreationTimestamp="2026-06-02 19:31:03 +0000 UTC" firstStartedPulling="2026-06-02 19:31:03.944282101 +0000 UTC m=+594.034150315" lastFinishedPulling="2026-06-02 19:31:05.191038971 +0000 UTC m=+595.280907186" observedRunningTime="2026-06-02 19:31:05.716340687 +0000 UTC m=+595.806208919" watchObservedRunningTime="2026-06-02 19:31:05.717837698 +0000 UTC m=+595.807705930" Jun 02 19:31:10.323846 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:10.323819 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:31:10.324282 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:10.324044 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:31:10.328028 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:10.328005 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:31:10.328572 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:10.328550 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:31:15.699409 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:15.699378 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:16.703399 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:16.703361 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-q7cxn" Jun 02 19:31:33.259775 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.259684 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz"] Jun 02 19:31:33.260267 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.259994 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" podUID="9e7218be-fbc0-4295-bf48-331d1de0fe9d" containerName="manager" containerID="cri-o://7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357" gracePeriod=10 Jun 02 19:31:33.510374 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.510297 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:33.527383 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.527346 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt"] Jun 02 19:31:33.527799 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.527785 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e7218be-fbc0-4295-bf48-331d1de0fe9d" containerName="manager" Jun 02 19:31:33.527850 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.527801 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e7218be-fbc0-4295-bf48-331d1de0fe9d" containerName="manager" Jun 02 19:31:33.527850 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.527823 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" containerName="manager" Jun 02 19:31:33.527850 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.527828 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" containerName="manager" Jun 02 19:31:33.527945 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.527887 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="9e7218be-fbc0-4295-bf48-331d1de0fe9d" containerName="manager" Jun 02 19:31:33.527945 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.527898 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="f7de0e1e-eda4-43d0-a9a3-123403baf57c" containerName="manager" Jun 02 19:31:33.531025 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.531007 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:33.548522 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.548475 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt"] Jun 02 19:31:33.700116 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.700074 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhz6m\" (UniqueName: \"kubernetes.io/projected/9e7218be-fbc0-4295-bf48-331d1de0fe9d-kube-api-access-rhz6m\") pod \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\" (UID: \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\") " Jun 02 19:31:33.700325 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.700154 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e7218be-fbc0-4295-bf48-331d1de0fe9d-extensions-socket-volume\") pod \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\" (UID: \"9e7218be-fbc0-4295-bf48-331d1de0fe9d\") " Jun 02 19:31:33.700325 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.700313 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vssqm\" (UniqueName: \"kubernetes.io/projected/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-kube-api-access-vssqm\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hdxbt\" (UID: \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:33.700457 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.700338 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hdxbt\" (UID: \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:33.700633 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.700606 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7218be-fbc0-4295-bf48-331d1de0fe9d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9e7218be-fbc0-4295-bf48-331d1de0fe9d" (UID: "9e7218be-fbc0-4295-bf48-331d1de0fe9d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:31:33.702236 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.702214 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e7218be-fbc0-4295-bf48-331d1de0fe9d-kube-api-access-rhz6m" (OuterVolumeSpecName: "kube-api-access-rhz6m") pod "9e7218be-fbc0-4295-bf48-331d1de0fe9d" (UID: "9e7218be-fbc0-4295-bf48-331d1de0fe9d"). InnerVolumeSpecName "kube-api-access-rhz6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:31:33.801332 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.801216 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vssqm\" (UniqueName: \"kubernetes.io/projected/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-kube-api-access-vssqm\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hdxbt\" (UID: \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:33.801332 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.801269 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hdxbt\" (UID: \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:33.801640 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.801390 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhz6m\" (UniqueName: \"kubernetes.io/projected/9e7218be-fbc0-4295-bf48-331d1de0fe9d-kube-api-access-rhz6m\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:31:33.801640 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.801400 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e7218be-fbc0-4295-bf48-331d1de0fe9d-extensions-socket-volume\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:31:33.801748 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.801727 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hdxbt\" (UID: \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:33.818572 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.818537 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vssqm\" (UniqueName: \"kubernetes.io/projected/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-kube-api-access-vssqm\") pod \"kuadrant-operator-controller-manager-55c7f4c975-hdxbt\" (UID: \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:33.825574 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.825541 2552 generic.go:358] "Generic (PLEG): container finished" podID="9e7218be-fbc0-4295-bf48-331d1de0fe9d" containerID="7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357" exitCode=0 Jun 02 19:31:33.825698 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.825600 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" Jun 02 19:31:33.825698 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.825618 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" event={"ID":"9e7218be-fbc0-4295-bf48-331d1de0fe9d","Type":"ContainerDied","Data":"7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357"} Jun 02 19:31:33.825698 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.825659 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz" event={"ID":"9e7218be-fbc0-4295-bf48-331d1de0fe9d","Type":"ContainerDied","Data":"e7484c1fbdbdaa5922ad7ee3bd945ac09f99cc89b81437faaea0fab5d961cc55"} Jun 02 19:31:33.825698 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.825676 2552 scope.go:117] "RemoveContainer" containerID="7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357" Jun 02 19:31:33.835123 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.835107 2552 scope.go:117] "RemoveContainer" containerID="7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357" Jun 02 19:31:33.835368 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:31:33.835352 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357\": container with ID starting with 7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357 not found: ID does not exist" containerID="7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357" Jun 02 19:31:33.835413 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.835375 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357"} err="failed to get container status \"7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357\": rpc error: code = NotFound desc = could not find container \"7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357\": container with ID starting with 7fb5d2a8241010c607bfa17fcede5d252718d1698cc9a7d066a8fdd14cf05357 not found: ID does not exist" Jun 02 19:31:33.841713 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.841697 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:33.879792 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.879750 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz"] Jun 02 19:31:33.893696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:33.893657 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-d7scz"] Jun 02 19:31:34.011262 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:34.011233 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt"] Jun 02 19:31:34.014242 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:31:34.014217 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2ab1b3_910c_4eff_bfff_8d2d246cbf1c.slice/crio-d6fae29526931c84f0cee08a33602195220a5458aa15225c71cb741f5bf31ce6 WatchSource:0}: Error finding container d6fae29526931c84f0cee08a33602195220a5458aa15225c71cb741f5bf31ce6: Status 404 returned error can't find the container with id d6fae29526931c84f0cee08a33602195220a5458aa15225c71cb741f5bf31ce6 Jun 02 19:31:34.394546 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:34.394487 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e7218be-fbc0-4295-bf48-331d1de0fe9d" path="/var/lib/kubelet/pods/9e7218be-fbc0-4295-bf48-331d1de0fe9d/volumes" Jun 02 19:31:34.832954 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:34.832919 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" event={"ID":"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c","Type":"ContainerStarted","Data":"2e68a73b9c2192d647aab385bcb86bf298d519ff29eb6c4979763d0ae86e4048"} Jun 02 19:31:34.833145 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:34.832970 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" event={"ID":"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c","Type":"ContainerStarted","Data":"d6fae29526931c84f0cee08a33602195220a5458aa15225c71cb741f5bf31ce6"} Jun 02 19:31:34.833145 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:34.833009 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:31:34.853914 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:34.853854 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" podStartSLOduration=1.853834166 podStartE2EDuration="1.853834166s" podCreationTimestamp="2026-06-02 19:31:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:31:34.850871687 +0000 UTC m=+624.940739920" watchObservedRunningTime="2026-06-02 19:31:34.853834166 +0000 UTC m=+624.943702401" Jun 02 19:31:45.839799 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:31:45.839767 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:32:21.917507 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.917456 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn"] Jun 02 19:32:21.922816 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.922793 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:21.925734 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.925710 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 19:32:21.926774 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.926749 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qd585\"" Jun 02 19:32:21.927055 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.926805 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 19:32:21.930979 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.930798 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn"] Jun 02 19:32:21.951366 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.951330 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:21.951559 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.951429 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:21.951559 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:21.951508 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r4qv\" (UniqueName: \"kubernetes.io/projected/f310f436-8cb4-418b-bd79-23b4fe75b27a-kube-api-access-7r4qv\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:22.052744 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.052698 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:22.052937 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.052807 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:22.052937 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.052880 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7r4qv\" (UniqueName: \"kubernetes.io/projected/f310f436-8cb4-418b-bd79-23b4fe75b27a-kube-api-access-7r4qv\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:22.053144 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.053127 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:22.053185 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.053166 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:22.062143 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.062111 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r4qv\" (UniqueName: \"kubernetes.io/projected/f310f436-8cb4-418b-bd79-23b4fe75b27a-kube-api-access-7r4qv\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:22.234289 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.234191 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:22.373102 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.373073 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn"] Jun 02 19:32:22.375166 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:32:22.375128 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf310f436_8cb4_418b_bd79_23b4fe75b27a.slice/crio-914e113229a18653fb851cb0824e0ab35f07017909e90b92687642e747587ee8 WatchSource:0}: Error finding container 914e113229a18653fb851cb0824e0ab35f07017909e90b92687642e747587ee8: Status 404 returned error can't find the container with id 914e113229a18653fb851cb0824e0ab35f07017909e90b92687642e747587ee8 Jun 02 19:32:22.377029 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:22.377010 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:32:23.031234 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:23.031197 2552 generic.go:358] "Generic (PLEG): container finished" podID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerID="91046b19538b964f08a320d7f4fb3823127a9b3f86c2608e00700b8fea38c11b" exitCode=0 Jun 02 19:32:23.031713 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:23.031286 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" event={"ID":"f310f436-8cb4-418b-bd79-23b4fe75b27a","Type":"ContainerDied","Data":"91046b19538b964f08a320d7f4fb3823127a9b3f86c2608e00700b8fea38c11b"} Jun 02 19:32:23.031713 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:23.031331 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" event={"ID":"f310f436-8cb4-418b-bd79-23b4fe75b27a","Type":"ContainerStarted","Data":"914e113229a18653fb851cb0824e0ab35f07017909e90b92687642e747587ee8"} Jun 02 19:32:24.037410 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:24.037316 2552 generic.go:358] "Generic (PLEG): container finished" podID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerID="807c8c49f5a74c24c9481a2509683a88ba8747c16dd22bb85a853b5954d656fe" exitCode=0 Jun 02 19:32:24.037829 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:24.037398 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" event={"ID":"f310f436-8cb4-418b-bd79-23b4fe75b27a","Type":"ContainerDied","Data":"807c8c49f5a74c24c9481a2509683a88ba8747c16dd22bb85a853b5954d656fe"} Jun 02 19:32:25.043180 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:25.043143 2552 generic.go:358] "Generic (PLEG): container finished" podID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerID="22b55f2bf3d02841d22ea94609d4612334e0a43b5496c010cfa603993878efc3" exitCode=0 Jun 02 19:32:25.043602 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:25.043206 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" event={"ID":"f310f436-8cb4-418b-bd79-23b4fe75b27a","Type":"ContainerDied","Data":"22b55f2bf3d02841d22ea94609d4612334e0a43b5496c010cfa603993878efc3"} Jun 02 19:32:26.189415 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.189389 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:26.291311 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.291278 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-bundle\") pod \"f310f436-8cb4-418b-bd79-23b4fe75b27a\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " Jun 02 19:32:26.291521 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.291323 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-util\") pod \"f310f436-8cb4-418b-bd79-23b4fe75b27a\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " Jun 02 19:32:26.291521 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.291423 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7r4qv\" (UniqueName: \"kubernetes.io/projected/f310f436-8cb4-418b-bd79-23b4fe75b27a-kube-api-access-7r4qv\") pod \"f310f436-8cb4-418b-bd79-23b4fe75b27a\" (UID: \"f310f436-8cb4-418b-bd79-23b4fe75b27a\") " Jun 02 19:32:26.291904 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.291870 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-bundle" (OuterVolumeSpecName: "bundle") pod "f310f436-8cb4-418b-bd79-23b4fe75b27a" (UID: "f310f436-8cb4-418b-bd79-23b4fe75b27a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:32:26.293685 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.293658 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f310f436-8cb4-418b-bd79-23b4fe75b27a-kube-api-access-7r4qv" (OuterVolumeSpecName: "kube-api-access-7r4qv") pod "f310f436-8cb4-418b-bd79-23b4fe75b27a" (UID: "f310f436-8cb4-418b-bd79-23b4fe75b27a"). InnerVolumeSpecName "kube-api-access-7r4qv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:32:26.296160 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.296128 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-util" (OuterVolumeSpecName: "util") pod "f310f436-8cb4-418b-bd79-23b4fe75b27a" (UID: "f310f436-8cb4-418b-bd79-23b4fe75b27a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:32:26.392319 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.392278 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-bundle\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:32:26.392319 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.392313 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f310f436-8cb4-418b-bd79-23b4fe75b27a-util\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:32:26.392559 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:26.392328 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r4qv\" (UniqueName: \"kubernetes.io/projected/f310f436-8cb4-418b-bd79-23b4fe75b27a-kube-api-access-7r4qv\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:32:27.053414 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:27.053377 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" event={"ID":"f310f436-8cb4-418b-bd79-23b4fe75b27a","Type":"ContainerDied","Data":"914e113229a18653fb851cb0824e0ab35f07017909e90b92687642e747587ee8"} Jun 02 19:32:27.053414 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:27.053414 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="914e113229a18653fb851cb0824e0ab35f07017909e90b92687642e747587ee8" Jun 02 19:32:27.053668 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:27.053443 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a7068jkn" Jun 02 19:32:45.728877 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.728845 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:32:45.729381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.729247 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerName="pull" Jun 02 19:32:45.729381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.729258 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerName="pull" Jun 02 19:32:45.729381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.729274 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerName="util" Jun 02 19:32:45.729381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.729279 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerName="util" Jun 02 19:32:45.729381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.729286 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerName="extract" Jun 02 19:32:45.729381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.729292 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerName="extract" Jun 02 19:32:45.729381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.729361 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="f310f436-8cb4-418b-bd79-23b4fe75b27a" containerName="extract" Jun 02 19:32:45.732598 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.732578 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:32:45.735202 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.735182 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 19:32:45.735202 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.735191 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 19:32:45.735381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.735185 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-jkrq5\"" Jun 02 19:32:45.735381 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.735230 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 19:32:45.740744 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.740715 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:32:45.874213 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.874177 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4wkb\" (UniqueName: \"kubernetes.io/projected/19e85ff3-1874-4196-88e6-497838dff7bc-kube-api-access-h4wkb\") pod \"maas-keycloak-0\" (UID: \"19e85ff3-1874-4196-88e6-497838dff7bc\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:32:45.975194 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.975157 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h4wkb\" (UniqueName: \"kubernetes.io/projected/19e85ff3-1874-4196-88e6-497838dff7bc-kube-api-access-h4wkb\") pod \"maas-keycloak-0\" (UID: \"19e85ff3-1874-4196-88e6-497838dff7bc\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:32:45.984158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:45.984086 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4wkb\" (UniqueName: \"kubernetes.io/projected/19e85ff3-1874-4196-88e6-497838dff7bc-kube-api-access-h4wkb\") pod \"maas-keycloak-0\" (UID: \"19e85ff3-1874-4196-88e6-497838dff7bc\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:32:46.044330 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:46.044296 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:32:46.186549 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:46.186514 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:32:46.186876 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:32:46.186849 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e85ff3_1874_4196_88e6_497838dff7bc.slice/crio-da2ed25ff7ac63bf73733348805697bdcc21645d72d086db428d321f3dbc0d42 WatchSource:0}: Error finding container da2ed25ff7ac63bf73733348805697bdcc21645d72d086db428d321f3dbc0d42: Status 404 returned error can't find the container with id da2ed25ff7ac63bf73733348805697bdcc21645d72d086db428d321f3dbc0d42 Jun 02 19:32:47.140884 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:47.140845 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"19e85ff3-1874-4196-88e6-497838dff7bc","Type":"ContainerStarted","Data":"da2ed25ff7ac63bf73733348805697bdcc21645d72d086db428d321f3dbc0d42"} Jun 02 19:32:52.165658 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:52.165603 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"19e85ff3-1874-4196-88e6-497838dff7bc","Type":"ContainerStarted","Data":"76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8"} Jun 02 19:32:52.188087 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:52.188019 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.569089472 podStartE2EDuration="7.187998581s" podCreationTimestamp="2026-06-02 19:32:45 +0000 UTC" firstStartedPulling="2026-06-02 19:32:46.188037006 +0000 UTC m=+696.277905217" lastFinishedPulling="2026-06-02 19:32:51.806946114 +0000 UTC m=+701.896814326" observedRunningTime="2026-06-02 19:32:52.185801979 +0000 UTC m=+702.275670212" watchObservedRunningTime="2026-06-02 19:32:52.187998581 +0000 UTC m=+702.277866817" Jun 02 19:32:53.045029 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:53.044977 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 19:32:53.046779 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:53.046747 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:32:54.045123 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:54.045069 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:32:55.045748 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:55.045694 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:32:56.044742 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:56.044695 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 19:32:56.045065 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:56.045023 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:32:57.046235 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:57.045790 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:32:58.045551 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:58.045394 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:32:59.045427 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:32:59.045376 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:33:00.044947 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:00.044898 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:33:01.045565 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:01.045475 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:33:02.045074 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:02.045004 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:33:03.045161 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:03.045095 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:33:04.044844 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:04.044794 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:33:05.045290 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:05.045233 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.49:9000/health/started\": dial tcp 10.133.0.49:9000: connect: connection refused" Jun 02 19:33:06.182250 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:06.182196 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 19:33:06.202164 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:06.201853 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 19:33:16.188435 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:16.188398 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 19:33:21.975043 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:21.975000 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c8fd668dd-2dhjf"] Jun 02 19:33:21.986150 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:21.986122 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:21.990257 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:21.990229 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cx544\"" Jun 02 19:33:21.990436 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:21.990351 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 19:33:21.996956 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:21.996927 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8fd668dd-2dhjf"] Jun 02 19:33:22.063415 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.063376 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpksx\" (UniqueName: \"kubernetes.io/projected/6a3c72d3-a3ff-4ebf-951e-0e936c159614-kube-api-access-hpksx\") pod \"maas-controller-6c8fd668dd-2dhjf\" (UID: \"6a3c72d3-a3ff-4ebf-951e-0e936c159614\") " pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:22.093929 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.093884 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-mwbk2"] Jun 02 19:33:22.097766 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.097742 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:33:22.107187 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.107160 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-mwbk2"] Jun 02 19:33:22.164570 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.164532 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hpksx\" (UniqueName: \"kubernetes.io/projected/6a3c72d3-a3ff-4ebf-951e-0e936c159614-kube-api-access-hpksx\") pod \"maas-controller-6c8fd668dd-2dhjf\" (UID: \"6a3c72d3-a3ff-4ebf-951e-0e936c159614\") " pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:22.173660 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.173632 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpksx\" (UniqueName: \"kubernetes.io/projected/6a3c72d3-a3ff-4ebf-951e-0e936c159614-kube-api-access-hpksx\") pod \"maas-controller-6c8fd668dd-2dhjf\" (UID: \"6a3c72d3-a3ff-4ebf-951e-0e936c159614\") " pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:22.266126 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.266023 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m54qk\" (UniqueName: \"kubernetes.io/projected/7e2be6c5-89d6-4ef3-add8-9ab2f9f78633-kube-api-access-m54qk\") pod \"maas-controller-6b5fb74b87-mwbk2\" (UID: \"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633\") " pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:33:22.297553 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.297509 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:22.367827 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.367483 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m54qk\" (UniqueName: \"kubernetes.io/projected/7e2be6c5-89d6-4ef3-add8-9ab2f9f78633-kube-api-access-m54qk\") pod \"maas-controller-6b5fb74b87-mwbk2\" (UID: \"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633\") " pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:33:22.376095 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.376067 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m54qk\" (UniqueName: \"kubernetes.io/projected/7e2be6c5-89d6-4ef3-add8-9ab2f9f78633-kube-api-access-m54qk\") pod \"maas-controller-6b5fb74b87-mwbk2\" (UID: \"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633\") " pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:33:22.408960 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.408926 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:33:22.444526 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.444285 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8fd668dd-2dhjf"] Jun 02 19:33:22.451528 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:33:22.451472 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3c72d3_a3ff_4ebf_951e_0e936c159614.slice/crio-1b35e23ec3707e28dffde7e66394d562e5c3e30f358f6653ea43b78861c155f5 WatchSource:0}: Error finding container 1b35e23ec3707e28dffde7e66394d562e5c3e30f358f6653ea43b78861c155f5: Status 404 returned error can't find the container with id 1b35e23ec3707e28dffde7e66394d562e5c3e30f358f6653ea43b78861c155f5 Jun 02 19:33:22.552315 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:22.552290 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-mwbk2"] Jun 02 19:33:22.554071 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:33:22.554041 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2be6c5_89d6_4ef3_add8_9ab2f9f78633.slice/crio-d78b8b768f6e23aafd248994b7da96454af89346509063a2abfb66f11f58c096 WatchSource:0}: Error finding container d78b8b768f6e23aafd248994b7da96454af89346509063a2abfb66f11f58c096: Status 404 returned error can't find the container with id d78b8b768f6e23aafd248994b7da96454af89346509063a2abfb66f11f58c096 Jun 02 19:33:23.365252 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:23.365212 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" event={"ID":"6a3c72d3-a3ff-4ebf-951e-0e936c159614","Type":"ContainerStarted","Data":"1b35e23ec3707e28dffde7e66394d562e5c3e30f358f6653ea43b78861c155f5"} Jun 02 19:33:23.366999 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:23.366965 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" event={"ID":"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633","Type":"ContainerStarted","Data":"d78b8b768f6e23aafd248994b7da96454af89346509063a2abfb66f11f58c096"} Jun 02 19:33:25.377589 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:25.377544 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" event={"ID":"6a3c72d3-a3ff-4ebf-951e-0e936c159614","Type":"ContainerStarted","Data":"0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018"} Jun 02 19:33:25.378076 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:25.377656 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:25.379356 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:25.379332 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" event={"ID":"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633","Type":"ContainerStarted","Data":"8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666"} Jun 02 19:33:25.379525 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:25.379448 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:33:25.406573 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:25.406512 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" podStartSLOduration=1.624829234 podStartE2EDuration="4.406475122s" podCreationTimestamp="2026-06-02 19:33:21 +0000 UTC" firstStartedPulling="2026-06-02 19:33:22.452701001 +0000 UTC m=+732.542569211" lastFinishedPulling="2026-06-02 19:33:25.234346877 +0000 UTC m=+735.324215099" observedRunningTime="2026-06-02 19:33:25.405824031 +0000 UTC m=+735.495692265" watchObservedRunningTime="2026-06-02 19:33:25.406475122 +0000 UTC m=+735.496343354" Jun 02 19:33:25.433552 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:25.433457 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" podStartSLOduration=0.742543292 podStartE2EDuration="3.433440656s" podCreationTimestamp="2026-06-02 19:33:22 +0000 UTC" firstStartedPulling="2026-06-02 19:33:22.555470587 +0000 UTC m=+732.645338798" lastFinishedPulling="2026-06-02 19:33:25.246367942 +0000 UTC m=+735.336236162" observedRunningTime="2026-06-02 19:33:25.430167863 +0000 UTC m=+735.520036124" watchObservedRunningTime="2026-06-02 19:33:25.433440656 +0000 UTC m=+735.523308889" Jun 02 19:33:36.394294 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:36.394265 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:36.394729 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:36.394308 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:33:36.454023 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:36.453984 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8fd668dd-2dhjf"] Jun 02 19:33:36.454239 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:36.454212 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" podUID="6a3c72d3-a3ff-4ebf-951e-0e936c159614" containerName="manager" containerID="cri-o://0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018" gracePeriod=30 Jun 02 19:33:36.713888 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:36.713862 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:36.809548 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:36.809510 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpksx\" (UniqueName: \"kubernetes.io/projected/6a3c72d3-a3ff-4ebf-951e-0e936c159614-kube-api-access-hpksx\") pod \"6a3c72d3-a3ff-4ebf-951e-0e936c159614\" (UID: \"6a3c72d3-a3ff-4ebf-951e-0e936c159614\") " Jun 02 19:33:36.811849 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:36.811816 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3c72d3-a3ff-4ebf-951e-0e936c159614-kube-api-access-hpksx" (OuterVolumeSpecName: "kube-api-access-hpksx") pod "6a3c72d3-a3ff-4ebf-951e-0e936c159614" (UID: "6a3c72d3-a3ff-4ebf-951e-0e936c159614"). InnerVolumeSpecName "kube-api-access-hpksx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:33:36.911203 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:36.911163 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpksx\" (UniqueName: \"kubernetes.io/projected/6a3c72d3-a3ff-4ebf-951e-0e936c159614-kube-api-access-hpksx\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:33:37.436133 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.436096 2552 generic.go:358] "Generic (PLEG): container finished" podID="6a3c72d3-a3ff-4ebf-951e-0e936c159614" containerID="0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018" exitCode=0 Jun 02 19:33:37.436663 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.436166 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" Jun 02 19:33:37.436663 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.436173 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" event={"ID":"6a3c72d3-a3ff-4ebf-951e-0e936c159614","Type":"ContainerDied","Data":"0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018"} Jun 02 19:33:37.436663 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.436213 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8fd668dd-2dhjf" event={"ID":"6a3c72d3-a3ff-4ebf-951e-0e936c159614","Type":"ContainerDied","Data":"1b35e23ec3707e28dffde7e66394d562e5c3e30f358f6653ea43b78861c155f5"} Jun 02 19:33:37.436663 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.436229 2552 scope.go:117] "RemoveContainer" containerID="0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018" Jun 02 19:33:37.445581 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.445560 2552 scope.go:117] "RemoveContainer" containerID="0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018" Jun 02 19:33:37.445857 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:33:37.445838 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018\": container with ID starting with 0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018 not found: ID does not exist" containerID="0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018" Jun 02 19:33:37.445908 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.445869 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018"} err="failed to get container status \"0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018\": rpc error: code = NotFound desc = could not find container \"0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018\": container with ID starting with 0f822ca7524b5e6af188176d8a34d78d09a79aba5a5cd88f449234caf7f11018 not found: ID does not exist" Jun 02 19:33:37.461170 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.461135 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8fd668dd-2dhjf"] Jun 02 19:33:37.481372 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:37.481341 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c8fd668dd-2dhjf"] Jun 02 19:33:38.395731 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:38.395695 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a3c72d3-a3ff-4ebf-951e-0e936c159614" path="/var/lib/kubelet/pods/6a3c72d3-a3ff-4ebf-951e-0e936c159614/volumes" Jun 02 19:33:56.419760 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.419725 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-699448d7c5-g2bbj"] Jun 02 19:33:56.420345 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.420329 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a3c72d3-a3ff-4ebf-951e-0e936c159614" containerName="manager" Jun 02 19:33:56.420391 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.420348 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a3c72d3-a3ff-4ebf-951e-0e936c159614" containerName="manager" Jun 02 19:33:56.420519 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.420483 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="6a3c72d3-a3ff-4ebf-951e-0e936c159614" containerName="manager" Jun 02 19:33:56.428989 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.428959 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:33:56.431065 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.431038 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-699448d7c5-g2bbj"] Jun 02 19:33:56.431599 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.431581 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 02 19:33:56.502895 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.502858 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8np2k\" (UniqueName: \"kubernetes.io/projected/4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0-kube-api-access-8np2k\") pod \"maas-api-699448d7c5-g2bbj\" (UID: \"4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0\") " pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:33:56.503061 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.503018 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0-maas-api-tls\") pod \"maas-api-699448d7c5-g2bbj\" (UID: \"4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0\") " pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:33:56.604157 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.604121 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0-maas-api-tls\") pod \"maas-api-699448d7c5-g2bbj\" (UID: \"4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0\") " pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:33:56.604330 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.604168 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8np2k\" (UniqueName: \"kubernetes.io/projected/4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0-kube-api-access-8np2k\") pod \"maas-api-699448d7c5-g2bbj\" (UID: \"4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0\") " pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:33:56.606574 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.606547 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0-maas-api-tls\") pod \"maas-api-699448d7c5-g2bbj\" (UID: \"4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0\") " pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:33:56.613664 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.613636 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8np2k\" (UniqueName: \"kubernetes.io/projected/4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0-kube-api-access-8np2k\") pod \"maas-api-699448d7c5-g2bbj\" (UID: \"4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0\") " pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:33:56.745388 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.745308 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:33:56.886167 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.886141 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-699448d7c5-g2bbj"] Jun 02 19:33:56.887631 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:33:56.887606 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac9de8e_dd2c_4f7c_91cc_7708d4022ff0.slice/crio-49ede4ea9594f2124ccd1f960e90e87a7f2bf53bf76c1002b2b95ecb7fc21a4b WatchSource:0}: Error finding container 49ede4ea9594f2124ccd1f960e90e87a7f2bf53bf76c1002b2b95ecb7fc21a4b: Status 404 returned error can't find the container with id 49ede4ea9594f2124ccd1f960e90e87a7f2bf53bf76c1002b2b95ecb7fc21a4b Jun 02 19:33:56.947888 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.947852 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-7x564"] Jun 02 19:33:56.953684 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.953657 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" Jun 02 19:33:56.958460 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.958430 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 02 19:33:56.958634 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.958475 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-v4vg6\"" Jun 02 19:33:56.961473 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:56.961450 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-7x564"] Jun 02 19:33:57.108475 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:57.108435 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chz4d\" (UniqueName: \"kubernetes.io/projected/944006fd-7391-4dc5-bad3-e3ff432f3c4f-kube-api-access-chz4d\") pod \"payload-processing-58b6c8fdc7-7x564\" (UID: \"944006fd-7391-4dc5-bad3-e3ff432f3c4f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" Jun 02 19:33:57.209445 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:57.209396 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-chz4d\" (UniqueName: \"kubernetes.io/projected/944006fd-7391-4dc5-bad3-e3ff432f3c4f-kube-api-access-chz4d\") pod \"payload-processing-58b6c8fdc7-7x564\" (UID: \"944006fd-7391-4dc5-bad3-e3ff432f3c4f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" Jun 02 19:33:57.218012 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:57.217975 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chz4d\" (UniqueName: \"kubernetes.io/projected/944006fd-7391-4dc5-bad3-e3ff432f3c4f-kube-api-access-chz4d\") pod \"payload-processing-58b6c8fdc7-7x564\" (UID: \"944006fd-7391-4dc5-bad3-e3ff432f3c4f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" Jun 02 19:33:57.267407 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:57.267371 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" Jun 02 19:33:57.403631 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:57.403544 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-7x564"] Jun 02 19:33:57.406586 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:33:57.406556 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944006fd_7391_4dc5_bad3_e3ff432f3c4f.slice/crio-55beff16035184fcf4f87f28799bafd615024e9db9e361e83c662543e696c41b WatchSource:0}: Error finding container 55beff16035184fcf4f87f28799bafd615024e9db9e361e83c662543e696c41b: Status 404 returned error can't find the container with id 55beff16035184fcf4f87f28799bafd615024e9db9e361e83c662543e696c41b Jun 02 19:33:57.528331 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:57.528291 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-699448d7c5-g2bbj" event={"ID":"4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0","Type":"ContainerStarted","Data":"49ede4ea9594f2124ccd1f960e90e87a7f2bf53bf76c1002b2b95ecb7fc21a4b"} Jun 02 19:33:57.529596 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:33:57.529568 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" event={"ID":"944006fd-7391-4dc5-bad3-e3ff432f3c4f","Type":"ContainerStarted","Data":"55beff16035184fcf4f87f28799bafd615024e9db9e361e83c662543e696c41b"} Jun 02 19:34:00.546602 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:00.546554 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" event={"ID":"944006fd-7391-4dc5-bad3-e3ff432f3c4f","Type":"ContainerStarted","Data":"83a84329e772b2520ce592f5b7a552358d625a0cbbb5708cf76eb463fdcdc819"} Jun 02 19:34:00.547103 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:00.546734 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" Jun 02 19:34:00.548089 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:00.548058 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-699448d7c5-g2bbj" event={"ID":"4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0","Type":"ContainerStarted","Data":"403a28a10214c649ff71992a027001a40b8de77d43f02ced079dc90187312f1c"} Jun 02 19:34:00.548233 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:00.548176 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:34:00.586257 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:00.586203 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" podStartSLOduration=2.180478143 podStartE2EDuration="4.586188707s" podCreationTimestamp="2026-06-02 19:33:56 +0000 UTC" firstStartedPulling="2026-06-02 19:33:57.408349857 +0000 UTC m=+767.498218071" lastFinishedPulling="2026-06-02 19:33:59.814060411 +0000 UTC m=+769.903928635" observedRunningTime="2026-06-02 19:34:00.583541942 +0000 UTC m=+770.673410176" watchObservedRunningTime="2026-06-02 19:34:00.586188707 +0000 UTC m=+770.676056939" Jun 02 19:34:00.603032 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:00.602950 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-699448d7c5-g2bbj" podStartSLOduration=1.7272232600000001 podStartE2EDuration="4.602930966s" podCreationTimestamp="2026-06-02 19:33:56 +0000 UTC" firstStartedPulling="2026-06-02 19:33:56.889201298 +0000 UTC m=+766.979069513" lastFinishedPulling="2026-06-02 19:33:59.764909003 +0000 UTC m=+769.854777219" observedRunningTime="2026-06-02 19:34:00.602479232 +0000 UTC m=+770.692347464" watchObservedRunningTime="2026-06-02 19:34:00.602930966 +0000 UTC m=+770.692799199" Jun 02 19:34:06.558464 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:06.558431 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-699448d7c5-g2bbj" Jun 02 19:34:08.416094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:08.416057 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:34:08.416522 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:08.416287 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" containerID="cri-o://76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8" gracePeriod=30 Jun 02 19:34:10.098001 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.097930 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.146302 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.146268 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4wkb\" (UniqueName: \"kubernetes.io/projected/19e85ff3-1874-4196-88e6-497838dff7bc-kube-api-access-h4wkb\") pod \"19e85ff3-1874-4196-88e6-497838dff7bc\" (UID: \"19e85ff3-1874-4196-88e6-497838dff7bc\") " Jun 02 19:34:10.148476 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.148446 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19e85ff3-1874-4196-88e6-497838dff7bc-kube-api-access-h4wkb" (OuterVolumeSpecName: "kube-api-access-h4wkb") pod "19e85ff3-1874-4196-88e6-497838dff7bc" (UID: "19e85ff3-1874-4196-88e6-497838dff7bc"). InnerVolumeSpecName "kube-api-access-h4wkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:34:10.247715 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.247673 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4wkb\" (UniqueName: \"kubernetes.io/projected/19e85ff3-1874-4196-88e6-497838dff7bc-kube-api-access-h4wkb\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:34:10.594229 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.594188 2552 generic.go:358] "Generic (PLEG): container finished" podID="19e85ff3-1874-4196-88e6-497838dff7bc" containerID="76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8" exitCode=143 Jun 02 19:34:10.594446 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.594252 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.594446 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.594279 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"19e85ff3-1874-4196-88e6-497838dff7bc","Type":"ContainerDied","Data":"76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8"} Jun 02 19:34:10.594446 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.594328 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"19e85ff3-1874-4196-88e6-497838dff7bc","Type":"ContainerDied","Data":"da2ed25ff7ac63bf73733348805697bdcc21645d72d086db428d321f3dbc0d42"} Jun 02 19:34:10.594446 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.594349 2552 scope.go:117] "RemoveContainer" containerID="76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8" Jun 02 19:34:10.604185 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.604147 2552 scope.go:117] "RemoveContainer" containerID="76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8" Jun 02 19:34:10.604448 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:34:10.604423 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8\": container with ID starting with 76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8 not found: ID does not exist" containerID="76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8" Jun 02 19:34:10.604555 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.604468 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8"} err="failed to get container status \"76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8\": rpc error: code = NotFound desc = could not find container \"76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8\": container with ID starting with 76e283019f7db58d45a312a0684a1981a0a35d7fbac56404cf72c84e1f3436e8 not found: ID does not exist" Jun 02 19:34:10.613793 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.613727 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:34:10.618541 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.618512 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:34:10.641910 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.641880 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:34:10.642369 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.642354 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" Jun 02 19:34:10.642413 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.642371 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" Jun 02 19:34:10.642472 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.642462 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" containerName="keycloak" Jun 02 19:34:10.646963 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.646948 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.650076 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.650042 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-jkrq5\"" Jun 02 19:34:10.650212 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.650114 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 19:34:10.650212 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.650137 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 19:34:10.650212 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.650141 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 19:34:10.650368 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.650121 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 02 19:34:10.656610 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.656584 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:34:10.753000 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.752964 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/5df5e294-ddce-4c2a-b9b0-8866ba2da3f0-test-realms\") pod \"maas-keycloak-0\" (UID: \"5df5e294-ddce-4c2a-b9b0-8866ba2da3f0\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.753216 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.753025 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b56gw\" (UniqueName: \"kubernetes.io/projected/5df5e294-ddce-4c2a-b9b0-8866ba2da3f0-kube-api-access-b56gw\") pod \"maas-keycloak-0\" (UID: \"5df5e294-ddce-4c2a-b9b0-8866ba2da3f0\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.854462 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.854424 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b56gw\" (UniqueName: \"kubernetes.io/projected/5df5e294-ddce-4c2a-b9b0-8866ba2da3f0-kube-api-access-b56gw\") pod \"maas-keycloak-0\" (UID: \"5df5e294-ddce-4c2a-b9b0-8866ba2da3f0\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.854696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.854571 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/5df5e294-ddce-4c2a-b9b0-8866ba2da3f0-test-realms\") pod \"maas-keycloak-0\" (UID: \"5df5e294-ddce-4c2a-b9b0-8866ba2da3f0\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.855218 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.855197 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/5df5e294-ddce-4c2a-b9b0-8866ba2da3f0-test-realms\") pod \"maas-keycloak-0\" (UID: \"5df5e294-ddce-4c2a-b9b0-8866ba2da3f0\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.864588 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.864533 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b56gw\" (UniqueName: \"kubernetes.io/projected/5df5e294-ddce-4c2a-b9b0-8866ba2da3f0-kube-api-access-b56gw\") pod \"maas-keycloak-0\" (UID: \"5df5e294-ddce-4c2a-b9b0-8866ba2da3f0\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:10.959368 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:10.959332 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:11.092565 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:11.092537 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:34:11.094473 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:34:11.094438 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df5e294_ddce_4c2a_b9b0_8866ba2da3f0.slice/crio-736c19e414f7237c3eb472284e0124f63dccd0c4edf8bac3ddf7aeec7c1efa03 WatchSource:0}: Error finding container 736c19e414f7237c3eb472284e0124f63dccd0c4edf8bac3ddf7aeec7c1efa03: Status 404 returned error can't find the container with id 736c19e414f7237c3eb472284e0124f63dccd0c4edf8bac3ddf7aeec7c1efa03 Jun 02 19:34:11.555464 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:11.555425 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-7x564" Jun 02 19:34:11.601180 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:11.601135 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5df5e294-ddce-4c2a-b9b0-8866ba2da3f0","Type":"ContainerStarted","Data":"736c19e414f7237c3eb472284e0124f63dccd0c4edf8bac3ddf7aeec7c1efa03"} Jun 02 19:34:12.396704 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:12.396671 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19e85ff3-1874-4196-88e6-497838dff7bc" path="/var/lib/kubelet/pods/19e85ff3-1874-4196-88e6-497838dff7bc/volumes" Jun 02 19:34:12.609831 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:12.609782 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5df5e294-ddce-4c2a-b9b0-8866ba2da3f0","Type":"ContainerStarted","Data":"9e239795ff5674bb5d600eed1e27b4eed0ae9af128500f52c02491873bc83434"} Jun 02 19:34:12.959937 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:12.959889 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:12.961592 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:12.961554 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:13.960577 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:13.960524 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:14.960395 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:14.960346 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:15.959924 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:15.959867 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:16.959922 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:16.959873 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:17.960936 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:17.960865 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:18.959829 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:18.959766 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:19.960130 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:19.960073 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:20.959946 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:20.959897 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:20.960339 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:20.960302 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:21.960464 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:21.960419 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:22.960352 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:22.960275 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:23.960285 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:23.960230 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:24.960413 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:24.960365 2552 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.54:9000/health/started\": dial tcp 10.133.0.54:9000: connect: connection refused" Jun 02 19:34:26.087979 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:26.087932 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 19:34:26.106480 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:26.106404 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="5df5e294-ddce-4c2a-b9b0-8866ba2da3f0" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 19:34:26.111878 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:26.111821 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=15.624998013 podStartE2EDuration="16.111803589s" podCreationTimestamp="2026-06-02 19:34:10 +0000 UTC" firstStartedPulling="2026-06-02 19:34:11.096052027 +0000 UTC m=+781.185920242" lastFinishedPulling="2026-06-02 19:34:11.582857589 +0000 UTC m=+781.672725818" observedRunningTime="2026-06-02 19:34:12.632978432 +0000 UTC m=+782.722846672" watchObservedRunningTime="2026-06-02 19:34:26.111803589 +0000 UTC m=+796.201671823" Jun 02 19:34:36.096264 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:34:36.096167 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 19:35:16.491569 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.491534 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6"] Jun 02 19:35:16.496746 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.496718 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.499745 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.499261 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fhmd9\"" Jun 02 19:35:16.499745 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.499575 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 19:35:16.500791 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.500764 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 02 19:35:16.500980 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.500945 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 19:35:16.507132 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.507103 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6"] Jun 02 19:35:16.551094 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.551048 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48xxw\" (UniqueName: \"kubernetes.io/projected/e476a67a-291d-4f37-8cc3-63c52f3d7502-kube-api-access-48xxw\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.551315 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.551126 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.551315 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.551177 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.551315 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.551217 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.551315 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.551295 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.551571 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.551413 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e476a67a-291d-4f37-8cc3-63c52f3d7502-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.651978 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.651939 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-48xxw\" (UniqueName: \"kubernetes.io/projected/e476a67a-291d-4f37-8cc3-63c52f3d7502-kube-api-access-48xxw\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.652158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.652007 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.652158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.652035 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.652158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.652060 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.652158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.652112 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.652366 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.652208 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e476a67a-291d-4f37-8cc3-63c52f3d7502-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.652553 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.652525 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.652664 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.652567 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.652664 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.652617 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.654732 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.654710 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e476a67a-291d-4f37-8cc3-63c52f3d7502-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.654962 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.654942 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e476a67a-291d-4f37-8cc3-63c52f3d7502-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.661902 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.661875 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48xxw\" (UniqueName: \"kubernetes.io/projected/e476a67a-291d-4f37-8cc3-63c52f3d7502-kube-api-access-48xxw\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6\" (UID: \"e476a67a-291d-4f37-8cc3-63c52f3d7502\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.812121 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.812023 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:16.958843 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:16.958816 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6"] Jun 02 19:35:16.961614 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:35:16.961578 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode476a67a_291d_4f37_8cc3_63c52f3d7502.slice/crio-3aa3074ad561b3789aa12cb4ee8c35e9cd9b42e0cfffdf84fed9373ca86e5f80 WatchSource:0}: Error finding container 3aa3074ad561b3789aa12cb4ee8c35e9cd9b42e0cfffdf84fed9373ca86e5f80: Status 404 returned error can't find the container with id 3aa3074ad561b3789aa12cb4ee8c35e9cd9b42e0cfffdf84fed9373ca86e5f80 Jun 02 19:35:17.910655 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:17.910609 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" event={"ID":"e476a67a-291d-4f37-8cc3-63c52f3d7502","Type":"ContainerStarted","Data":"3aa3074ad561b3789aa12cb4ee8c35e9cd9b42e0cfffdf84fed9373ca86e5f80"} Jun 02 19:35:20.925790 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:20.925744 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" event={"ID":"e476a67a-291d-4f37-8cc3-63c52f3d7502","Type":"ContainerStarted","Data":"3cada2a0829b9801588140837c61278a7ff615a1e39cee9a8675dbdb6fe2ce2f"} Jun 02 19:35:21.931613 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:21.931579 2552 generic.go:358] "Generic (PLEG): container finished" podID="e476a67a-291d-4f37-8cc3-63c52f3d7502" containerID="3cada2a0829b9801588140837c61278a7ff615a1e39cee9a8675dbdb6fe2ce2f" exitCode=0 Jun 02 19:35:21.931613 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:21.931619 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" event={"ID":"e476a67a-291d-4f37-8cc3-63c52f3d7502","Type":"ContainerDied","Data":"3cada2a0829b9801588140837c61278a7ff615a1e39cee9a8675dbdb6fe2ce2f"} Jun 02 19:35:24.948035 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:24.947993 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" event={"ID":"e476a67a-291d-4f37-8cc3-63c52f3d7502","Type":"ContainerStarted","Data":"70da8504f2ae30a30c77c51ac957b2bcef57e7121b4fb2cf3f13bb526020270b"} Jun 02 19:35:24.966756 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:24.966703 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" podStartSLOduration=1.9232679620000002 podStartE2EDuration="8.966686125s" podCreationTimestamp="2026-06-02 19:35:16 +0000 UTC" firstStartedPulling="2026-06-02 19:35:16.963826808 +0000 UTC m=+847.053695023" lastFinishedPulling="2026-06-02 19:35:24.007244967 +0000 UTC m=+854.097113186" observedRunningTime="2026-06-02 19:35:24.96581492 +0000 UTC m=+855.055683154" watchObservedRunningTime="2026-06-02 19:35:24.966686125 +0000 UTC m=+855.056554358" Jun 02 19:35:26.812433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:26.812392 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:26.812433 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:26.812438 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:26.825373 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:26.825337 2552 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:35:26.973344 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:35:26.973311 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6" Jun 02 19:36:10.358901 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:36:10.358868 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:36:10.368680 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:36:10.368643 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:36:10.370641 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:36:10.370613 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:36:10.372946 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:36:10.372921 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:39:31.922348 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:31.922303 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-mwbk2"] Jun 02 19:39:31.923004 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:31.922656 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" podUID="7e2be6c5-89d6-4ef3-add8-9ab2f9f78633" containerName="manager" containerID="cri-o://8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666" gracePeriod=30 Jun 02 19:39:32.181877 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:32.181805 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:39:32.286696 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:32.286651 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m54qk\" (UniqueName: \"kubernetes.io/projected/7e2be6c5-89d6-4ef3-add8-9ab2f9f78633-kube-api-access-m54qk\") pod \"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633\" (UID: \"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633\") " Jun 02 19:39:32.288847 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:32.288811 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2be6c5-89d6-4ef3-add8-9ab2f9f78633-kube-api-access-m54qk" (OuterVolumeSpecName: "kube-api-access-m54qk") pod "7e2be6c5-89d6-4ef3-add8-9ab2f9f78633" (UID: "7e2be6c5-89d6-4ef3-add8-9ab2f9f78633"). InnerVolumeSpecName "kube-api-access-m54qk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:39:32.387707 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:32.387664 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m54qk\" (UniqueName: \"kubernetes.io/projected/7e2be6c5-89d6-4ef3-add8-9ab2f9f78633-kube-api-access-m54qk\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:39:33.027635 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.027599 2552 generic.go:358] "Generic (PLEG): container finished" podID="7e2be6c5-89d6-4ef3-add8-9ab2f9f78633" containerID="8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666" exitCode=0 Jun 02 19:39:33.028092 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.027681 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" Jun 02 19:39:33.028092 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.027691 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" event={"ID":"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633","Type":"ContainerDied","Data":"8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666"} Jun 02 19:39:33.028092 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.027738 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b5fb74b87-mwbk2" event={"ID":"7e2be6c5-89d6-4ef3-add8-9ab2f9f78633","Type":"ContainerDied","Data":"d78b8b768f6e23aafd248994b7da96454af89346509063a2abfb66f11f58c096"} Jun 02 19:39:33.028092 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.027758 2552 scope.go:117] "RemoveContainer" containerID="8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666" Jun 02 19:39:33.037755 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.037730 2552 scope.go:117] "RemoveContainer" containerID="8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666" Jun 02 19:39:33.038090 ip-10-0-138-231 kubenswrapper[2552]: E0602 19:39:33.038065 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666\": container with ID starting with 8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666 not found: ID does not exist" containerID="8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666" Jun 02 19:39:33.038178 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.038101 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666"} err="failed to get container status \"8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666\": rpc error: code = NotFound desc = could not find container \"8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666\": container with ID starting with 8c9b20a70ed105861fc1f182356871259e2f5aac8ab6ddcefc54c2115ce20666 not found: ID does not exist" Jun 02 19:39:33.046001 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.045967 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-mwbk2"] Jun 02 19:39:33.049359 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.049331 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-mwbk2"] Jun 02 19:39:33.798583 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.798548 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-rpgg8"] Jun 02 19:39:33.799055 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.799041 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e2be6c5-89d6-4ef3-add8-9ab2f9f78633" containerName="manager" Jun 02 19:39:33.799100 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.799057 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e2be6c5-89d6-4ef3-add8-9ab2f9f78633" containerName="manager" Jun 02 19:39:33.799136 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.799123 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="7e2be6c5-89d6-4ef3-add8-9ab2f9f78633" containerName="manager" Jun 02 19:39:33.803367 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.803349 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" Jun 02 19:39:33.806021 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.806003 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cx544\"" Jun 02 19:39:33.806138 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.806007 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 19:39:33.821462 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.821431 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-rpgg8"] Jun 02 19:39:33.901724 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:33.901679 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5vcg\" (UniqueName: \"kubernetes.io/projected/f52e24c9-b275-4edc-a174-6fe71bd966cb-kube-api-access-p5vcg\") pod \"maas-controller-6b5fb74b87-rpgg8\" (UID: \"f52e24c9-b275-4edc-a174-6fe71bd966cb\") " pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" Jun 02 19:39:34.003014 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:34.002971 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5vcg\" (UniqueName: \"kubernetes.io/projected/f52e24c9-b275-4edc-a174-6fe71bd966cb-kube-api-access-p5vcg\") pod \"maas-controller-6b5fb74b87-rpgg8\" (UID: \"f52e24c9-b275-4edc-a174-6fe71bd966cb\") " pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" Jun 02 19:39:34.011880 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:34.011851 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5vcg\" (UniqueName: \"kubernetes.io/projected/f52e24c9-b275-4edc-a174-6fe71bd966cb-kube-api-access-p5vcg\") pod \"maas-controller-6b5fb74b87-rpgg8\" (UID: \"f52e24c9-b275-4edc-a174-6fe71bd966cb\") " pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" Jun 02 19:39:34.123226 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:34.123178 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" Jun 02 19:39:34.266667 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:34.266633 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b5fb74b87-rpgg8"] Jun 02 19:39:34.268322 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:39:34.268291 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52e24c9_b275_4edc_a174_6fe71bd966cb.slice/crio-e31d4f702a58f195c04a94363111689eef1f056edf54df00f12451d7a8b082a2 WatchSource:0}: Error finding container e31d4f702a58f195c04a94363111689eef1f056edf54df00f12451d7a8b082a2: Status 404 returned error can't find the container with id e31d4f702a58f195c04a94363111689eef1f056edf54df00f12451d7a8b082a2 Jun 02 19:39:34.269645 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:34.269625 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:39:34.396302 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:34.396212 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e2be6c5-89d6-4ef3-add8-9ab2f9f78633" path="/var/lib/kubelet/pods/7e2be6c5-89d6-4ef3-add8-9ab2f9f78633/volumes" Jun 02 19:39:35.039211 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:35.039102 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" event={"ID":"f52e24c9-b275-4edc-a174-6fe71bd966cb","Type":"ContainerStarted","Data":"34b29fde9e2f50f3092fe17b7dfa624eea034ce9e63e5f69127c8403337edc66"} Jun 02 19:39:35.039211 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:35.039147 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" event={"ID":"f52e24c9-b275-4edc-a174-6fe71bd966cb","Type":"ContainerStarted","Data":"e31d4f702a58f195c04a94363111689eef1f056edf54df00f12451d7a8b082a2"} Jun 02 19:39:35.039466 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:35.039275 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" Jun 02 19:39:35.057053 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:35.056997 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" podStartSLOduration=1.568748968 podStartE2EDuration="2.056980575s" podCreationTimestamp="2026-06-02 19:39:33 +0000 UTC" firstStartedPulling="2026-06-02 19:39:34.269753295 +0000 UTC m=+1104.359621506" lastFinishedPulling="2026-06-02 19:39:34.757984902 +0000 UTC m=+1104.847853113" observedRunningTime="2026-06-02 19:39:35.054306935 +0000 UTC m=+1105.144175170" watchObservedRunningTime="2026-06-02 19:39:35.056980575 +0000 UTC m=+1105.146848858" Jun 02 19:39:46.049272 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:39:46.049234 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b5fb74b87-rpgg8" Jun 02 19:41:10.401610 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:41:10.401580 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:41:10.405529 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:41:10.405504 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:41:10.405919 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:41:10.405900 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:41:10.409661 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:41:10.409644 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:46:10.435280 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:46:10.435248 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:46:10.439442 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:46:10.439422 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:46:10.443255 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:46:10.443237 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:46:10.447614 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:46:10.447594 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:49:59.387045 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.386953 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt"] Jun 02 19:49:59.387619 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.387222 2552 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" podUID="7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c" containerName="manager" containerID="cri-o://2e68a73b9c2192d647aab385bcb86bf298d519ff29eb6c4979763d0ae86e4048" gracePeriod=10 Jun 02 19:49:59.637518 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.637406 2552 generic.go:358] "Generic (PLEG): container finished" podID="7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c" containerID="2e68a73b9c2192d647aab385bcb86bf298d519ff29eb6c4979763d0ae86e4048" exitCode=0 Jun 02 19:49:59.637518 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.637467 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" event={"ID":"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c","Type":"ContainerDied","Data":"2e68a73b9c2192d647aab385bcb86bf298d519ff29eb6c4979763d0ae86e4048"} Jun 02 19:49:59.751424 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.751402 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:49:59.764237 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.764217 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vssqm\" (UniqueName: \"kubernetes.io/projected/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-kube-api-access-vssqm\") pod \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\" (UID: \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\") " Jun 02 19:49:59.764334 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.764308 2552 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-extensions-socket-volume\") pod \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\" (UID: \"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c\") " Jun 02 19:49:59.764695 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.764670 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c" (UID: "7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:49:59.766198 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.766177 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-kube-api-access-vssqm" (OuterVolumeSpecName: "kube-api-access-vssqm") pod "7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c" (UID: "7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c"). InnerVolumeSpecName "kube-api-access-vssqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:49:59.865698 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.865658 2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-extensions-socket-volume\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:49:59.865698 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:49:59.865690 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vssqm\" (UniqueName: \"kubernetes.io/projected/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c-kube-api-access-vssqm\") on node \"ip-10-0-138-231.ec2.internal\" DevicePath \"\"" Jun 02 19:50:00.642658 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:50:00.642625 2552 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" Jun 02 19:50:00.643097 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:50:00.642628 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt" event={"ID":"7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c","Type":"ContainerDied","Data":"d6fae29526931c84f0cee08a33602195220a5458aa15225c71cb741f5bf31ce6"} Jun 02 19:50:00.643097 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:50:00.642752 2552 scope.go:117] "RemoveContainer" containerID="2e68a73b9c2192d647aab385bcb86bf298d519ff29eb6c4979763d0ae86e4048" Jun 02 19:50:00.664570 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:50:00.664532 2552 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt"] Jun 02 19:50:00.670976 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:50:00.670951 2552 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-hdxbt"] Jun 02 19:50:02.394957 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:50:02.394923 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c" path="/var/lib/kubelet/pods/7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c/volumes" Jun 02 19:51:05.535158 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.535120 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp"] Jun 02 19:51:05.535690 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.535599 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c" containerName="manager" Jun 02 19:51:05.535690 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.535613 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c" containerName="manager" Jun 02 19:51:05.535813 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.535697 2552 memory_manager.go:356] "RemoveStaleState removing state" podUID="7a2ab1b3-910c-4eff-bfff-8d2d246cbf1c" containerName="manager" Jun 02 19:51:05.538905 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.538888 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:05.541569 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.541549 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dg7pj\"" Jun 02 19:51:05.549486 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.549462 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp"] Jun 02 19:51:05.668852 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.668805 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdrjr\" (UniqueName: \"kubernetes.io/projected/d341c6ac-c571-4a92-b268-2c7f9337873b-kube-api-access-gdrjr\") pod \"kuadrant-operator-controller-manager-55c7f4c975-j2wwp\" (UID: \"d341c6ac-c571-4a92-b268-2c7f9337873b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:05.669050 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.669013 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d341c6ac-c571-4a92-b268-2c7f9337873b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-j2wwp\" (UID: \"d341c6ac-c571-4a92-b268-2c7f9337873b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:05.770438 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.770393 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d341c6ac-c571-4a92-b268-2c7f9337873b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-j2wwp\" (UID: \"d341c6ac-c571-4a92-b268-2c7f9337873b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:05.770644 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.770461 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gdrjr\" (UniqueName: \"kubernetes.io/projected/d341c6ac-c571-4a92-b268-2c7f9337873b-kube-api-access-gdrjr\") pod \"kuadrant-operator-controller-manager-55c7f4c975-j2wwp\" (UID: \"d341c6ac-c571-4a92-b268-2c7f9337873b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:05.770819 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.770795 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d341c6ac-c571-4a92-b268-2c7f9337873b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-j2wwp\" (UID: \"d341c6ac-c571-4a92-b268-2c7f9337873b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:05.784012 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.783983 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdrjr\" (UniqueName: \"kubernetes.io/projected/d341c6ac-c571-4a92-b268-2c7f9337873b-kube-api-access-gdrjr\") pod \"kuadrant-operator-controller-manager-55c7f4c975-j2wwp\" (UID: \"d341c6ac-c571-4a92-b268-2c7f9337873b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:05.850971 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:05.850922 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:06.014138 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:06.014108 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp"] Jun 02 19:51:06.015167 ip-10-0-138-231 kubenswrapper[2552]: W0602 19:51:06.015134 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd341c6ac_c571_4a92_b268_2c7f9337873b.slice/crio-ede77ef9417c6c15bfaca11afeba24dcb48b0346d7cb35447a90d0c6ff032fd5 WatchSource:0}: Error finding container ede77ef9417c6c15bfaca11afeba24dcb48b0346d7cb35447a90d0c6ff032fd5: Status 404 returned error can't find the container with id ede77ef9417c6c15bfaca11afeba24dcb48b0346d7cb35447a90d0c6ff032fd5 Jun 02 19:51:06.017560 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:06.017542 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:51:06.910062 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:06.910028 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" event={"ID":"d341c6ac-c571-4a92-b268-2c7f9337873b","Type":"ContainerStarted","Data":"ab391b7b515008918232e2c8bbc4a295b07f4e835e6b8f1f7bd5a77b03c014bd"} Jun 02 19:51:06.910062 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:06.910066 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" event={"ID":"d341c6ac-c571-4a92-b268-2c7f9337873b","Type":"ContainerStarted","Data":"ede77ef9417c6c15bfaca11afeba24dcb48b0346d7cb35447a90d0c6ff032fd5"} Jun 02 19:51:06.910506 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:06.910089 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:51:06.935361 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:06.935314 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" podStartSLOduration=1.935299648 podStartE2EDuration="1.935299648s" podCreationTimestamp="2026-06-02 19:51:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:51:06.932288487 +0000 UTC m=+1797.022156766" watchObservedRunningTime="2026-06-02 19:51:06.935299648 +0000 UTC m=+1797.025167881" Jun 02 19:51:10.470959 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:10.470933 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:51:10.475138 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:10.475118 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:51:10.482041 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:10.482022 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:51:10.485429 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:10.485414 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:51:17.916353 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:51:17.916319 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-j2wwp" Jun 02 19:56:10.505308 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:56:10.505278 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:56:10.509687 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:56:10.509663 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 19:56:10.515780 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:56:10.515760 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 19:56:10.519806 ip-10-0-138-231 kubenswrapper[2552]: I0602 19:56:10.519789 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 20:01:10.549865 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:10.549752 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 20:01:10.554548 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:10.554521 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 20:01:10.558564 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:10.558540 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 20:01:10.562359 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:10.562341 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 20:01:16.476948 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:16.476919 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-79dcdb775-lqbbc_a4c79d26-173e-4ae9-abf6-c493746cdf25/nginx/0.log" Jun 02 20:01:16.583727 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:16.583697 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-k427x_05286ae9-7530-4264-928b-a175fd594e57/manager/0.log" Jun 02 20:01:16.723249 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:16.723219 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-kbnjs_1c44a5f1-62ae-4dc6-b425-86460cff13d1/manager/0.log" Jun 02 20:01:16.848476 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:16.848450 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-699448d7c5-g2bbj_4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0/maas-api/0.log" Jun 02 20:01:16.990330 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:16.990300 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b5fb74b87-rpgg8_f52e24c9-b275-4edc-a174-6fe71bd966cb/manager/0.log" Jun 02 20:01:17.101531 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:17.101434 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-8wwnl_7cd3146f-a731-43f0-b236-394302ed8baf/server/0.log" Jun 02 20:01:17.230510 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:17.230462 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-bd4dl_37f23b73-2276-47ef-9c7b-0dc0910007f1/manager/0.log" Jun 02 20:01:17.458981 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:17.458892 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b5889499b-s48jk_c2fb622a-a5fd-4129-92e5-fe19e9c72ae7/copy-manifests/0.log" Jun 02 20:01:17.466537 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:17.466485 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b5889499b-s48jk_c2fb622a-a5fd-4129-92e5-fe19e9c72ae7/manager/0.log" Jun 02 20:01:18.465107 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:18.465076 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b_184ffb75-84b8-4a0d-8905-f7894e460c8d/extract/0.log" Jun 02 20:01:18.470962 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:18.470941 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b_184ffb75-84b8-4a0d-8905-f7894e460c8d/util/0.log" Jun 02 20:01:18.477109 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:18.477087 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b_184ffb75-84b8-4a0d-8905-f7894e460c8d/pull/0.log" Jun 02 20:01:18.845701 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:18.845674 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4_33b106b5-189e-4cdd-a470-24a7aa180809/util/0.log" Jun 02 20:01:18.852076 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:18.852041 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4_33b106b5-189e-4cdd-a470-24a7aa180809/pull/0.log" Jun 02 20:01:18.858146 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:18.858129 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4_33b106b5-189e-4cdd-a470-24a7aa180809/extract/0.log" Jun 02 20:01:19.218824 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:19.218744 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-hz4qk_ac79c3e7-c694-418c-b958-2059ec5f5e3d/manager/0.log" Jun 02 20:01:19.573802 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:19.573772 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-j2wwp_d341c6ac-c571-4a92-b268-2c7f9337873b/manager/0.log" Jun 02 20:01:19.834537 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:19.834424 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-q7cxn_3eb2d116-9ca7-47b1-a788-7a02ff1c7a68/manager/0.log" Jun 02 20:01:20.346044 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:20.346015 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-xwqhh_7fde924f-ad81-40f6-b877-7d36e75e90c8/discovery/0.log" Jun 02 20:01:20.589308 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:20.589275 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-897b5b6bb-74tz2_c6785b6d-7b5b-4358-bf79-fbd38042ab29/kube-auth-proxy/0.log" Jun 02 20:01:20.843808 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:20.843771 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-7x564_944006fd-7391-4dc5-bad3-e3ff432f3c4f/payload-processing/0.log" Jun 02 20:01:20.959534 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:20.959485 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59d8544f99-c22p5_3a06f660-d2cd-419b-9b5c-32109faa93b4/router/0.log" Jun 02 20:01:21.573558 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:21.573528 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6_e476a67a-291d-4f37-8cc3-63c52f3d7502/storage-initializer/0.log" Jun 02 20:01:21.580965 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:21.580939 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-7kld6_e476a67a-291d-4f37-8cc3-63c52f3d7502/main/0.log" Jun 02 20:01:29.523523 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:29.523428 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tk8zj_222e8269-50c6-471a-80fa-a9aedc1574ec/global-pull-secret-syncer/0.log" Jun 02 20:01:29.701948 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:29.701916 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cxvvr_8a375452-9057-4bd7-9337-422d42b662e0/konnectivity-agent/0.log" Jun 02 20:01:29.754790 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:29.754756 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-231.ec2.internal_03340f265015a5ce3cd39b70624e15fb/haproxy/0.log" Jun 02 20:01:33.388544 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:33.388513 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b_184ffb75-84b8-4a0d-8905-f7894e460c8d/extract/0.log" Jun 02 20:01:33.409449 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:33.409426 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b_184ffb75-84b8-4a0d-8905-f7894e460c8d/util/0.log" Jun 02 20:01:33.430502 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:33.430460 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x8s5b_184ffb75-84b8-4a0d-8905-f7894e460c8d/pull/0.log" Jun 02 20:01:33.626936 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:33.626904 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4_33b106b5-189e-4cdd-a470-24a7aa180809/extract/0.log" Jun 02 20:01:33.646811 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:33.646744 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4_33b106b5-189e-4cdd-a470-24a7aa180809/util/0.log" Jun 02 20:01:33.672083 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:33.672057 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlqg4_33b106b5-189e-4cdd-a470-24a7aa180809/pull/0.log" Jun 02 20:01:34.169268 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:34.169234 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-hz4qk_ac79c3e7-c694-418c-b958-2059ec5f5e3d/manager/0.log" Jun 02 20:01:34.344935 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:34.344898 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-j2wwp_d341c6ac-c571-4a92-b268-2c7f9337873b/manager/0.log" Jun 02 20:01:34.462152 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:34.462060 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-q7cxn_3eb2d116-9ca7-47b1-a788-7a02ff1c7a68/manager/0.log" Jun 02 20:01:35.682322 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:35.682292 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3/alertmanager/0.log" Jun 02 20:01:35.712182 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:35.712154 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3/config-reloader/0.log" Jun 02 20:01:35.736242 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:35.736220 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3/kube-rbac-proxy-web/0.log" Jun 02 20:01:35.764214 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:35.764176 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3/kube-rbac-proxy/0.log" Jun 02 20:01:35.789165 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:35.789134 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3/kube-rbac-proxy-metric/0.log" Jun 02 20:01:35.814712 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:35.814688 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3/prom-label-proxy/0.log" Jun 02 20:01:35.840335 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:35.840305 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9f6fe7a8-541e-4225-bc33-a8aba2e8c6f3/init-config-reloader/0.log" Jun 02 20:01:36.009810 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.009721 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78c7859fc5-dsfjk_180b464f-74a6-41e3-8d53-89f87b51f55a/metrics-server/0.log" Jun 02 20:01:36.065717 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.065693 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlxgv_23cb6867-6cf0-4299-b352-74534ddc1c83/node-exporter/0.log" Jun 02 20:01:36.085659 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.085638 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlxgv_23cb6867-6cf0-4299-b352-74534ddc1c83/kube-rbac-proxy/0.log" Jun 02 20:01:36.105950 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.105933 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlxgv_23cb6867-6cf0-4299-b352-74534ddc1c83/init-textfile/0.log" Jun 02 20:01:36.326663 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.326634 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-j6zv2_9a361f5f-8ef3-47fb-b76b-4bff82d6c357/kube-rbac-proxy-main/0.log" Jun 02 20:01:36.352383 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.352360 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-j6zv2_9a361f5f-8ef3-47fb-b76b-4bff82d6c357/kube-rbac-proxy-self/0.log" Jun 02 20:01:36.374355 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.374333 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-j6zv2_9a361f5f-8ef3-47fb-b76b-4bff82d6c357/openshift-state-metrics/0.log" Jun 02 20:01:36.698464 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.698373 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b6488f97-mkgxp_7f511b0a-7259-4f91-88f4-ec4b262067e7/telemeter-client/0.log" Jun 02 20:01:36.722640 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.722616 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b6488f97-mkgxp_7f511b0a-7259-4f91-88f4-ec4b262067e7/reload/0.log" Jun 02 20:01:36.745551 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.745526 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b6488f97-mkgxp_7f511b0a-7259-4f91-88f4-ec4b262067e7/kube-rbac-proxy/0.log" Jun 02 20:01:36.773584 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.773557 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-94fcb5df7-285xl_b4889ba7-21a1-41ca-b62b-9432f10bb11f/thanos-query/0.log" Jun 02 20:01:36.795725 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.795698 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-94fcb5df7-285xl_b4889ba7-21a1-41ca-b62b-9432f10bb11f/kube-rbac-proxy-web/0.log" Jun 02 20:01:36.815746 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.815727 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-94fcb5df7-285xl_b4889ba7-21a1-41ca-b62b-9432f10bb11f/kube-rbac-proxy/0.log" Jun 02 20:01:36.852035 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.852013 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-94fcb5df7-285xl_b4889ba7-21a1-41ca-b62b-9432f10bb11f/prom-label-proxy/0.log" Jun 02 20:01:36.872089 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.872069 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-94fcb5df7-285xl_b4889ba7-21a1-41ca-b62b-9432f10bb11f/kube-rbac-proxy-rules/0.log" Jun 02 20:01:36.898078 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:36.898057 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-94fcb5df7-285xl_b4889ba7-21a1-41ca-b62b-9432f10bb11f/kube-rbac-proxy-metrics/0.log" Jun 02 20:01:38.543449 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:38.543418 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/2.log" Jun 02 20:01:38.551370 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:38.551342 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f5qhp_238f4e41-37e5-454b-84fb-12af646cc790/console-operator/3.log" Jun 02 20:01:39.062117 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:39.062089 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b578888bd-zx59h_2f233028-e8a5-4cbb-9159-44ecf7e41948/console/0.log" Jun 02 20:01:39.099324 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:39.099300 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-pgl2t_55a7d134-12a2-4bd6-8c88-4dd937c306c7/download-server/0.log" Jun 02 20:01:39.659158 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:39.659124 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7598fd5bd-w5k97_076f8656-2a48-47cf-8b9e-017008bab7ff/volume-data-source-validator/0.log" Jun 02 20:01:40.357354 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.357296 2552 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2"] Jun 02 20:01:40.361603 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.361583 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.363998 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.363980 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-chkxm\"/\"openshift-service-ca.crt\"" Jun 02 20:01:40.364121 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.364026 2552 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-chkxm\"/\"default-dockercfg-wmdjg\"" Jun 02 20:01:40.365190 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.365174 2552 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-chkxm\"/\"kube-root-ca.crt\"" Jun 02 20:01:40.369661 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.369634 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2"] Jun 02 20:01:40.440343 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.440308 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-podres\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.440574 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.440406 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8fwk\" (UniqueName: \"kubernetes.io/projected/72269efd-adcc-4a09-803a-0aa73eebdb8f-kube-api-access-h8fwk\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.440574 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.440449 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-lib-modules\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.440574 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.440480 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-sys\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.440739 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.440578 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-proc\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.541577 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.541546 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-proc\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.541732 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.541619 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-podres\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.541732 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.541654 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-proc\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.541732 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.541673 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h8fwk\" (UniqueName: \"kubernetes.io/projected/72269efd-adcc-4a09-803a-0aa73eebdb8f-kube-api-access-h8fwk\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.541898 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.541879 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-lib-modules\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.541950 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.541908 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-podres\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.541950 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.541928 2552 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-sys\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.542047 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.542030 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-sys\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.542096 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.542051 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72269efd-adcc-4a09-803a-0aa73eebdb8f-lib-modules\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.552868 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.552845 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8fwk\" (UniqueName: \"kubernetes.io/projected/72269efd-adcc-4a09-803a-0aa73eebdb8f-kube-api-access-h8fwk\") pod \"perf-node-gather-daemonset-2x9v2\" (UID: \"72269efd-adcc-4a09-803a-0aa73eebdb8f\") " pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.563364 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.563337 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5k45w_4ab4d72b-4346-4824-8058-f14f5de2fbab/dns/0.log" Jun 02 20:01:40.622428 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.622356 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5k45w_4ab4d72b-4346-4824-8058-f14f5de2fbab/kube-rbac-proxy/0.log" Jun 02 20:01:40.673828 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.673802 2552 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:40.812064 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.812036 2552 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2"] Jun 02 20:01:40.813200 ip-10-0-138-231 kubenswrapper[2552]: W0602 20:01:40.813172 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod72269efd_adcc_4a09_803a_0aa73eebdb8f.slice/crio-75352014c2f08b11799f4c83ed8acf39528493313be4cdf3342044e1b1c6ad7f WatchSource:0}: Error finding container 75352014c2f08b11799f4c83ed8acf39528493313be4cdf3342044e1b1c6ad7f: Status 404 returned error can't find the container with id 75352014c2f08b11799f4c83ed8acf39528493313be4cdf3342044e1b1c6ad7f Jun 02 20:01:40.815064 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.815048 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:01:40.878998 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:40.878928 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mc8fl_859620d1-906d-4775-bf20-d888549445d4/dns-node-resolver/0.log" Jun 02 20:01:41.386259 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:41.386220 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-98cf6659c-sfh8t_93d0c8df-f627-41d4-918c-0e60643554ec/registry/0.log" Jun 02 20:01:41.408047 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:41.408017 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-26fj5_98907b61-a541-4b22-8cd6-0f31f6baea96/node-ca/0.log" Jun 02 20:01:41.523861 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:41.523829 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" event={"ID":"72269efd-adcc-4a09-803a-0aa73eebdb8f","Type":"ContainerStarted","Data":"688f01a84b594ec670ba49dad65bb220f0c7e2fa3d737a2cf78e7aa40bd71f56"} Jun 02 20:01:41.523861 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:41.523864 2552 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" event={"ID":"72269efd-adcc-4a09-803a-0aa73eebdb8f","Type":"ContainerStarted","Data":"75352014c2f08b11799f4c83ed8acf39528493313be4cdf3342044e1b1c6ad7f"} Jun 02 20:01:41.524119 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:41.523890 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:41.543817 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:41.543755 2552 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" podStartSLOduration=1.543734572 podStartE2EDuration="1.543734572s" podCreationTimestamp="2026-06-02 20:01:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:01:41.540789757 +0000 UTC m=+2431.630657992" watchObservedRunningTime="2026-06-02 20:01:41.543734572 +0000 UTC m=+2431.633602804" Jun 02 20:01:42.466786 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:42.466752 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-xwqhh_7fde924f-ad81-40f6-b877-7d36e75e90c8/discovery/0.log" Jun 02 20:01:42.516314 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:42.516283 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-897b5b6bb-74tz2_c6785b6d-7b5b-4358-bf79-fbd38042ab29/kube-auth-proxy/0.log" Jun 02 20:01:42.685710 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:42.685678 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-7x564_944006fd-7391-4dc5-bad3-e3ff432f3c4f/payload-processing/0.log" Jun 02 20:01:42.707609 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:42.707569 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59d8544f99-c22p5_3a06f660-d2cd-419b-9b5c-32109faa93b4/router/0.log" Jun 02 20:01:43.407974 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:43.407934 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zb24h_6bb96f33-bcb4-4a7e-b879-159149ced68e/serve-healthcheck-canary/0.log" Jun 02 20:01:44.164352 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:44.164318 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r747v_e6dcacfc-140d-4bed-bb93-18a8436546fb/kube-rbac-proxy/0.log" Jun 02 20:01:44.186276 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:44.186250 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r747v_e6dcacfc-140d-4bed-bb93-18a8436546fb/exporter/0.log" Jun 02 20:01:44.210365 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:44.210335 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r747v_e6dcacfc-140d-4bed-bb93-18a8436546fb/extractor/0.log" Jun 02 20:01:46.232066 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.232037 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-79dcdb775-lqbbc_a4c79d26-173e-4ae9-abf6-c493746cdf25/nginx/0.log" Jun 02 20:01:46.255771 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.255741 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-k427x_05286ae9-7530-4264-928b-a175fd594e57/manager/0.log" Jun 02 20:01:46.380366 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.380339 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-kbnjs_1c44a5f1-62ae-4dc6-b425-86460cff13d1/manager/0.log" Jun 02 20:01:46.426429 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.426402 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-699448d7c5-g2bbj_4ac9de8e-dd2c-4f7c-91cc-7708d4022ff0/maas-api/0.log" Jun 02 20:01:46.536866 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.536773 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b5fb74b87-rpgg8_f52e24c9-b275-4edc-a174-6fe71bd966cb/manager/0.log" Jun 02 20:01:46.557673 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.557643 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-8wwnl_7cd3146f-a731-43f0-b236-394302ed8baf/server/0.log" Jun 02 20:01:46.631018 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.630986 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-bd4dl_37f23b73-2276-47ef-9c7b-0dc0910007f1/manager/0.log" Jun 02 20:01:46.698919 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.698878 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b5889499b-s48jk_c2fb622a-a5fd-4129-92e5-fe19e9c72ae7/manager/0.log" Jun 02 20:01:46.716810 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:46.716788 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b5889499b-s48jk_c2fb622a-a5fd-4129-92e5-fe19e9c72ae7/copy-manifests/0.log" Jun 02 20:01:47.538088 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:47.538060 2552 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-chkxm/perf-node-gather-daemonset-2x9v2" Jun 02 20:01:53.692287 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:53.692253 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-fk87w_e563216c-9d47-4016-b694-2fc740bccd5d/kube-storage-version-migrator-operator/1.log" Jun 02 20:01:53.693736 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:53.693708 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-fk87w_e563216c-9d47-4016-b694-2fc740bccd5d/kube-storage-version-migrator-operator/0.log" Jun 02 20:01:55.001326 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.001297 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxw6v_b739e36f-5df5-4cec-8f18-498e13c693f7/kube-multus-additional-cni-plugins/0.log" Jun 02 20:01:55.021826 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.021806 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxw6v_b739e36f-5df5-4cec-8f18-498e13c693f7/egress-router-binary-copy/0.log" Jun 02 20:01:55.041097 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.041077 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxw6v_b739e36f-5df5-4cec-8f18-498e13c693f7/cni-plugins/0.log" Jun 02 20:01:55.062287 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.062259 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxw6v_b739e36f-5df5-4cec-8f18-498e13c693f7/bond-cni-plugin/0.log" Jun 02 20:01:55.084145 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.084120 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxw6v_b739e36f-5df5-4cec-8f18-498e13c693f7/routeoverride-cni/0.log" Jun 02 20:01:55.105924 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.105905 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxw6v_b739e36f-5df5-4cec-8f18-498e13c693f7/whereabouts-cni-bincopy/0.log" Jun 02 20:01:55.131402 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.131379 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxw6v_b739e36f-5df5-4cec-8f18-498e13c693f7/whereabouts-cni/0.log" Jun 02 20:01:55.412093 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.412064 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wxwcg_8b7e9986-2590-42d6-8fba-e49e780ebc72/kube-multus/0.log" Jun 02 20:01:55.584206 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.584176 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sgj2b_386d5a57-cdc7-4d70-b3af-c06ea739c213/network-metrics-daemon/0.log" Jun 02 20:01:55.601527 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:55.601503 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sgj2b_386d5a57-cdc7-4d70-b3af-c06ea739c213/kube-rbac-proxy/0.log" Jun 02 20:01:56.491259 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.491229 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-controller/0.log" Jun 02 20:01:56.508993 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.508967 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/0.log" Jun 02 20:01:56.531530 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.531487 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovn-acl-logging/1.log" Jun 02 20:01:56.555364 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.555321 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/kube-rbac-proxy-node/0.log" Jun 02 20:01:56.579939 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.579908 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 20:01:56.595978 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.595926 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/northd/0.log" Jun 02 20:01:56.616122 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.616100 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/nbdb/0.log" Jun 02 20:01:56.636364 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.636344 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/sbdb/0.log" Jun 02 20:01:56.799799 ip-10-0-138-231 kubenswrapper[2552]: I0602 20:01:56.799718 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xppd_f8636918-e4cf-4852-b6e7-640039dca7b8/ovnkube-controller/0.log"