Jun 02 05:23:21.634161 ip-10-0-141-22 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 02 05:23:21.634170 ip-10-0-141-22 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 02 05:23:21.634177 ip-10-0-141-22 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 02 05:23:21.634406 ip-10-0-141-22 systemd[1]: Failed to start Kubernetes Kubelet. Jun 02 05:23:31.849233 ip-10-0-141-22 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 02 05:23:31.849255 ip-10-0-141-22 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a25d3c594fc24447a45601a38c9fca78 -- Jun 02 05:25:48.188240 ip-10-0-141-22 systemd[1]: Starting Kubernetes Kubelet... Jun 02 05:25:48.553703 ip-10-0-141-22 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 05:25:48.553703 ip-10-0-141-22 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 05:25:48.553703 ip-10-0-141-22 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 05:25:48.553703 ip-10-0-141-22 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 05:25:48.553703 ip-10-0-141-22 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 05:25:48.555043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.554946 2561 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 05:25:48.560171 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560156 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 05:25:48.560171 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560171 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560175 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560178 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560181 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560184 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560186 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560189 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560191 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560194 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560197 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560199 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560202 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560204 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560212 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560214 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560217 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560219 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560222 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560224 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 05:25:48.560234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560227 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560230 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560235 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560238 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560241 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560243 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560246 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560249 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560252 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560254 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560257 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560259 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560262 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560264 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560266 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560269 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560271 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560274 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560276 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560278 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 05:25:48.560684 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560281 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560284 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560286 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560289 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560291 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560295 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560299 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560301 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560304 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560307 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560310 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560313 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560316 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560319 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560323 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560327 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560330 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560332 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560335 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560337 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 05:25:48.561224 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560340 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560342 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560345 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560347 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560350 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560352 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560356 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560358 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560361 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560363 2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560367 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560369 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560372 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560374 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560376 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560379 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560381 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560383 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560386 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560389 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 05:25:48.561786 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560391 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560394 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560397 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560399 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560402 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560404 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560798 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560803 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560806 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560808 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560811 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560813 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560817 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560819 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560822 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560824 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560827 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560829 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560832 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560835 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 05:25:48.562286 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560838 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560840 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560843 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560845 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560848 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560850 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560853 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560855 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560858 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560860 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560863 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560866 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560868 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560871 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560873 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560877 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560884 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560886 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560889 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560891 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 02 05:25:48.562785 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560894 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560896 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560899 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560901 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560903 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560906 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560908 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560911 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560914 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560916 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560919 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560922 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560925 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560927 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560930 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560933 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560935 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560938 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560940 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560943 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 05:25:48.563291 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560945 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560948 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560950 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560953 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560957 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560977 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560980 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560983 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560987 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560990 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560993 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560995 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.560998 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561000 2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561003 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561005 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561008 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561010 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561013 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 05:25:48.563836 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561015 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561017 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561020 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561022 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561024 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561028 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561030 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561033 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561035 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561038 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561040 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561043 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561045 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561115 2561 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561126 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561132 2561 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561136 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561140 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561143 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561147 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561151 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 05:25:48.564326 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561156 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561159 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561162 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561167 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561170 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561173 2561 flags.go:64] FLAG: --cgroup-root="" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561176 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561180 2561 flags.go:64] FLAG: --client-ca-file="" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561182 2561 flags.go:64] FLAG: --cloud-config="" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561185 2561 flags.go:64] FLAG: --cloud-provider="external" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561188 2561 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561191 2561 flags.go:64] FLAG: --cluster-domain="" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561194 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561197 2561 flags.go:64] FLAG: --config-dir="" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561200 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561203 2561 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561207 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561211 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561214 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561217 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561220 2561 flags.go:64] FLAG: --contention-profiling="false" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561223 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561226 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561229 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561232 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 05:25:48.564829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561236 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561239 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561242 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561245 2561 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561247 2561 flags.go:64] FLAG: --enable-server="true" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561250 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561255 2561 flags.go:64] FLAG: --event-burst="100" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561259 2561 flags.go:64] FLAG: --event-qps="50" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561261 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561264 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561268 2561 flags.go:64] FLAG: --eviction-hard="" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561271 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561274 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561277 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561280 2561 flags.go:64] FLAG: --eviction-soft="" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561282 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561285 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561288 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561291 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561294 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561296 2561 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561299 2561 flags.go:64] FLAG: --feature-gates="" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561302 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561305 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561308 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 05:25:48.565465 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561312 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561315 2561 flags.go:64] FLAG: --healthz-port="10248" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561317 2561 flags.go:64] FLAG: --help="false" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561320 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561323 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561326 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561328 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561331 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561335 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561337 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561340 2561 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561342 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561345 2561 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561348 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561351 2561 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561354 2561 flags.go:64] FLAG: --kube-reserved="" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561357 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561360 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561363 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561366 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561369 2561 flags.go:64] FLAG: --lock-file="" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561372 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561375 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561377 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 05:25:48.566074 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561382 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561385 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561388 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561391 2561 flags.go:64] FLAG: --logging-format="text" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561394 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561397 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561399 2561 flags.go:64] FLAG: --manifest-url="" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561402 2561 flags.go:64] FLAG: --manifest-url-header="" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561406 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561410 2561 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561414 2561 flags.go:64] FLAG: --max-pods="110" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561417 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561419 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561422 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561425 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561428 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561431 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561434 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561441 2561 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561444 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561447 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561450 2561 flags.go:64] FLAG: --pod-cidr="" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561452 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 05:25:48.566659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561458 2561 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561461 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561464 2561 flags.go:64] FLAG: --pods-per-core="0" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561467 2561 flags.go:64] FLAG: --port="10250" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561471 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561473 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02a672797cb77d878" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561477 2561 flags.go:64] FLAG: --qos-reserved="" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561479 2561 flags.go:64] FLAG: --read-only-port="10255" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561482 2561 flags.go:64] FLAG: --register-node="true" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561485 2561 flags.go:64] FLAG: --register-schedulable="true" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561488 2561 flags.go:64] FLAG: --register-with-taints="" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561491 2561 flags.go:64] FLAG: --registry-burst="10" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561494 2561 flags.go:64] FLAG: --registry-qps="5" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561497 2561 flags.go:64] FLAG: --reserved-cpus="" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561499 2561 flags.go:64] FLAG: --reserved-memory="" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561503 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561506 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561509 2561 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561512 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561515 2561 flags.go:64] FLAG: --runonce="false" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561518 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561521 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561524 2561 flags.go:64] FLAG: --seccomp-default="false" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561527 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561529 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561532 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 05:25:48.567235 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561535 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561538 2561 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561541 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561543 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561546 2561 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561549 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561552 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561555 2561 flags.go:64] FLAG: --system-cgroups="" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561558 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561563 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561566 2561 flags.go:64] FLAG: --tls-cert-file="" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561569 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561572 2561 flags.go:64] FLAG: --tls-min-version="" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561575 2561 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561578 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561581 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561583 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561586 2561 flags.go:64] FLAG: --v="2" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561590 2561 flags.go:64] FLAG: --version="false" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561594 2561 flags.go:64] FLAG: --vmodule="" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561598 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.561601 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561710 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561715 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 05:25:48.568179 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561718 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561721 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561729 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561731 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561734 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561737 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561739 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561742 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561744 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561746 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561749 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561751 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561754 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561757 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561760 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561763 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561766 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561769 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561771 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 05:25:48.569203 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561774 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561776 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561779 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561781 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561784 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561786 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561789 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561792 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561795 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561797 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561800 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561802 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561805 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561807 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561809 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561811 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561820 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561822 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561825 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561827 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 05:25:48.570001 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561829 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561832 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561834 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561836 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561839 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561841 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561846 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561849 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561853 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561856 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561859 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561862 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561865 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561868 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561870 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561873 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561875 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561877 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561880 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 05:25:48.570907 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561882 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561884 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561887 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561889 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561892 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561894 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561896 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561899 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561901 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561904 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561912 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561914 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561917 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561919 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561921 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561924 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561926 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561929 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561931 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561935 2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 05:25:48.571456 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561938 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561940 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561944 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561947 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561949 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.561952 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.562646 2561 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.569601 2561 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.569742 2561 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569815 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569822 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569827 2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569832 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569837 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569842 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 05:25:48.572061 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569846 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569851 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569855 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569860 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569864 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569868 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569873 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569877 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569882 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569886 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569891 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569895 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569899 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569904 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569908 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569912 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569919 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569928 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569933 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569938 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 02 05:25:48.572628 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569942 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569947 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569951 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569955 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569978 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569985 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569990 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569994 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.569998 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570002 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570006 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570010 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570014 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570018 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570022 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570026 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570030 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570035 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570039 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 05:25:48.573486 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570043 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570047 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570051 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570056 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570060 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570064 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570068 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570072 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570077 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570081 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570086 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570090 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570094 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570098 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570102 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570107 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570111 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570115 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570121 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570127 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 05:25:48.574085 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570132 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570136 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570140 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570145 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570149 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570153 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570157 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570188 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570195 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570200 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570204 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570209 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570213 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570218 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570222 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570227 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570231 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570236 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570242 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570248 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 05:25:48.574715 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570252 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.570261 2561 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570419 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570427 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570432 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570437 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570442 2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570446 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570451 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570455 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570459 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570464 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570469 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570473 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570478 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570482 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 05:25:48.575234 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570486 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570490 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570495 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570499 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570503 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570506 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570510 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570515 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570521 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570526 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570530 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570534 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570538 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570542 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570547 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570550 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570554 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570559 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570564 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570568 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 05:25:48.575731 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570572 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570576 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570580 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570584 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570589 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570592 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570597 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570601 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570605 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570610 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570614 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570618 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570623 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570627 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570631 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570635 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570639 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570643 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570647 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570651 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 05:25:48.576397 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570655 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570659 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570663 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570667 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570672 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570676 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570680 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570684 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570689 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570693 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570698 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570702 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570715 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570722 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570727 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570732 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570736 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570740 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570745 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 05:25:48.576920 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570750 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570754 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570759 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570765 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570769 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570773 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570777 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570781 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570785 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570789 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570793 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570797 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:48.570801 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.570809 2561 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.571642 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 05:25:48.577435 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.574251 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 05:25:48.577814 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.575357 2561 server.go:1019] "Starting client certificate rotation" Jun 02 05:25:48.577814 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.575461 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 05:25:48.577814 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.575502 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 05:25:48.598452 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.598430 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 05:25:48.600618 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.600599 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 05:25:48.612814 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.612799 2561 log.go:25] "Validated CRI v1 runtime API" Jun 02 05:25:48.617800 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.617786 2561 log.go:25] "Validated CRI v1 image API" Jun 02 05:25:48.619186 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.619168 2561 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 05:25:48.624089 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.624071 2561 fs.go:135] Filesystem UUIDs: map[2dff13a6-ec23-4075-81f3-136f0fda8ae0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9d708384-3bc7-4de1-9208-b7d6c5500000:/dev/nvme0n1p3] Jun 02 05:25:48.624153 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.624090 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 05:25:48.627338 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.627322 2561 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 05:25:48.628159 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.628053 2561 manager.go:217] Machine: {Timestamp:2026-06-02 05:25:48.627567006 +0000 UTC m=+0.338016243 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099304 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec273281d20aea9f4e2a7efbf55eefef SystemUUID:ec273281-d20a-ea9f-4e2a-7efbf55eefef BootID:a25d3c59-4fc2-4447-a456-01a38c9fca78 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a7:2c:8a:0c:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a7:2c:8a:0c:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:49:9a:1c:1f:02 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 05:25:48.628720 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.628710 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 05:25:48.628797 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.628786 2561 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 05:25:48.630459 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.630437 2561 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 05:25:48.630639 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.630463 2561 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-22.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 05:25:48.630692 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.630650 2561 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 05:25:48.630692 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.630658 2561 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 05:25:48.630692 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.630665 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 05:25:48.631283 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.631273 2561 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 05:25:48.632595 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.632585 2561 state_mem.go:36] "Initialized new in-memory state store" Jun 02 05:25:48.632705 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.632697 2561 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 05:25:48.634608 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.634598 2561 kubelet.go:491] "Attempting to sync node with API server" Jun 02 05:25:48.634648 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.634616 2561 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 05:25:48.634648 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.634628 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 05:25:48.634648 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.634637 2561 kubelet.go:397] "Adding apiserver pod source" Jun 02 05:25:48.634648 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.634646 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 05:25:48.635607 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.635596 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 05:25:48.635642 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.635615 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 05:25:48.638749 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.638695 2561 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 05:25:48.640762 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.640747 2561 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 05:25:48.642800 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642788 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 05:25:48.642845 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642810 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 05:25:48.642845 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642818 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 05:25:48.642845 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642824 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 05:25:48.642845 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642832 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 05:25:48.642845 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642838 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 05:25:48.642845 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642844 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 05:25:48.643032 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642849 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 05:25:48.643032 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642857 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 05:25:48.643032 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642863 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 05:25:48.643032 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642871 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 05:25:48.643032 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.642877 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 05:25:48.643586 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.643577 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 05:25:48.643618 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.643587 2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 05:25:48.646921 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.646903 2561 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 05:25:48.647019 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.646919 2561 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 05:25:48.647019 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.646942 2561 server.go:1295] "Started kubelet" Jun 02 05:25:48.647115 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.647012 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 05:25:48.647115 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.647036 2561 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 05:25:48.647115 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.647098 2561 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 05:25:48.647387 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.647368 2561 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-22.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 05:25:48.647477 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.647456 2561 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-22.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 05:25:48.647788 ip-10-0-141-22 systemd[1]: Started Kubernetes Kubelet. Jun 02 05:25:48.649230 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.649218 2561 server.go:317] "Adding debug handlers to kubelet server" Jun 02 05:25:48.650815 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.650796 2561 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 05:25:48.653773 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.653750 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ckfnc" Jun 02 05:25:48.654415 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.654387 2561 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 05:25:48.654642 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.653848 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-22.ec2.internal.18b52c4fbb09c396 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-22.ec2.internal,UID:ip-10-0-141-22.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-22.ec2.internal,},FirstTimestamp:2026-06-02 05:25:48.646917014 +0000 UTC m=+0.357366254,LastTimestamp:2026-06-02 05:25:48.646917014 +0000 UTC m=+0.357366254,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-22.ec2.internal,}" Jun 02 05:25:48.656079 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.656064 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 05:25:48.656523 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.656496 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 05:25:48.656604 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.656554 2561 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 05:25:48.656604 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.656565 2561 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 05:25:48.656690 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.656646 2561 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 05:25:48.656739 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.656708 2561 reconstruct.go:97] "Volume reconstruction finished" Jun 02 05:25:48.656739 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.656717 2561 reconciler.go:26] "Reconciler: start to sync state" Jun 02 05:25:48.657252 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.657227 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:48.657350 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.657335 2561 factory.go:55] Registering systemd factory Jun 02 05:25:48.657417 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.657357 2561 factory.go:223] Registration of the systemd container factory successfully Jun 02 05:25:48.657568 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.657557 2561 factory.go:153] Registering CRI-O factory Jun 02 05:25:48.657605 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.657579 2561 factory.go:223] Registration of the crio container factory successfully Jun 02 05:25:48.657637 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.657625 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 05:25:48.657666 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.657640 2561 factory.go:103] Registering Raw factory Jun 02 05:25:48.657666 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.657650 2561 manager.go:1196] Started watching for new ooms in manager Jun 02 05:25:48.658071 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.658055 2561 manager.go:319] Starting recovery of all containers Jun 02 05:25:48.661122 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.661089 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ckfnc" Jun 02 05:25:48.668453 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.668421 2561 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 05:25:48.670942 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.670917 2561 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-22.ec2.internal\" not found" node="ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.674048 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.674032 2561 manager.go:324] Recovery completed Jun 02 05:25:48.677915 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.677902 2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 05:25:48.679576 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.679561 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientMemory" Jun 02 05:25:48.679636 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.679589 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 05:25:48.679636 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.679600 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientPID" Jun 02 05:25:48.680115 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.680103 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 05:25:48.680115 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.680113 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 05:25:48.680200 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.680130 2561 state_mem.go:36] "Initialized new in-memory state store" Jun 02 05:25:48.682311 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.682298 2561 policy_none.go:49] "None policy: Start" Jun 02 05:25:48.682370 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.682316 2561 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 05:25:48.682370 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.682326 2561 state_mem.go:35] "Initializing new in-memory state store" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.722142 2561 manager.go:341] "Starting Device Plugin manager" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.722194 2561 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.722203 2561 server.go:85] "Starting device plugin registration server" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.722410 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.722430 2561 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.722563 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.722650 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.722657 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.723317 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 05:25:48.723426 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.723360 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:48.780201 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.780163 2561 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 05:25:48.781498 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.781483 2561 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 05:25:48.781557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.781513 2561 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 05:25:48.781557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.781533 2561 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 05:25:48.781557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.781542 2561 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 05:25:48.781677 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.781613 2561 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 05:25:48.785717 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.785699 2561 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 05:25:48.822710 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.822658 2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 05:25:48.823597 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.823581 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientMemory" Jun 02 05:25:48.823665 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.823612 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 05:25:48.823665 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.823623 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientPID" Jun 02 05:25:48.823665 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.823644 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.831849 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.831833 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.831918 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.831854 2561 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-22.ec2.internal\": node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:48.850907 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.850887 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:48.882279 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.882257 2561 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal"] Jun 02 05:25:48.882359 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.882341 2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 05:25:48.883282 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.883269 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientMemory" Jun 02 05:25:48.883337 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.883295 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 05:25:48.883337 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.883304 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientPID" Jun 02 05:25:48.884503 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.884492 2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 05:25:48.884651 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.884637 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.884697 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.884665 2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 05:25:48.885203 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.885189 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientMemory" Jun 02 05:25:48.885284 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.885208 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 05:25:48.885284 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.885217 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientPID" Jun 02 05:25:48.885284 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.885195 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientMemory" Jun 02 05:25:48.885284 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.885244 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 05:25:48.885284 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.885255 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientPID" Jun 02 05:25:48.886266 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.886253 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.886309 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.886275 2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 05:25:48.888634 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.888615 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientMemory" Jun 02 05:25:48.888723 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.888647 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 05:25:48.888723 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.888658 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeHasSufficientPID" Jun 02 05:25:48.914493 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.914476 2561 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-22.ec2.internal\" not found" node="ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.918923 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.918907 2561 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-22.ec2.internal\" not found" node="ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.952012 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:48.951989 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:48.958818 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.958800 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8560f6e96c5dd0d18e85416cd362a94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal\" (UID: \"d8560f6e96c5dd0d18e85416cd362a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.958888 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.958826 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8560f6e96c5dd0d18e85416cd362a94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal\" (UID: \"d8560f6e96c5dd0d18e85416cd362a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:48.958888 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:48.958845 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/165b8d8bc20dd92d757a5571670d3d63-config\") pod \"kube-apiserver-proxy-ip-10-0-141-22.ec2.internal\" (UID: \"165b8d8bc20dd92d757a5571670d3d63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.052754 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.052711 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.059012 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.058991 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8560f6e96c5dd0d18e85416cd362a94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal\" (UID: \"d8560f6e96c5dd0d18e85416cd362a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.059080 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.059021 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8560f6e96c5dd0d18e85416cd362a94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal\" (UID: \"d8560f6e96c5dd0d18e85416cd362a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.059080 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.059038 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/165b8d8bc20dd92d757a5571670d3d63-config\") pod \"kube-apiserver-proxy-ip-10-0-141-22.ec2.internal\" (UID: \"165b8d8bc20dd92d757a5571670d3d63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.059080 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.059063 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/165b8d8bc20dd92d757a5571670d3d63-config\") pod \"kube-apiserver-proxy-ip-10-0-141-22.ec2.internal\" (UID: \"165b8d8bc20dd92d757a5571670d3d63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.059174 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.059087 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8560f6e96c5dd0d18e85416cd362a94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal\" (UID: \"d8560f6e96c5dd0d18e85416cd362a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.059174 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.059095 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8560f6e96c5dd0d18e85416cd362a94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal\" (UID: \"d8560f6e96c5dd0d18e85416cd362a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.153480 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.153397 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.216888 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.216854 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.222928 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.222912 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.253745 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.253716 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.354235 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.354192 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.454756 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.454670 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.555226 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.555197 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.575586 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.575558 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 05:25:49.575732 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.575711 2561 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 05:25:49.575781 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.575711 2561 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 05:25:49.655551 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.655528 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.656615 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.656599 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 05:25:49.663637 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.663339 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 05:20:48 +0000 UTC" deadline="2027-11-08 11:52:25.327165224 +0000 UTC" Jun 02 05:25:49.663637 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.663369 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12582h26m35.663800022s" Jun 02 05:25:49.667168 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.667151 2561 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 05:25:49.743365 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:49.743324 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165b8d8bc20dd92d757a5571670d3d63.slice/crio-0b62453661fc1fb5546de8d5b896285c523211d74f71b04248608dfe79de7532 WatchSource:0}: Error finding container 0b62453661fc1fb5546de8d5b896285c523211d74f71b04248608dfe79de7532: Status 404 returned error can't find the container with id 0b62453661fc1fb5546de8d5b896285c523211d74f71b04248608dfe79de7532 Jun 02 05:25:49.743533 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:49.743518 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8560f6e96c5dd0d18e85416cd362a94.slice/crio-ec99ba591fb1b2e14ec194d93ee3f2a5adab090c2805cf90d748b9ec6c7cb3bd WatchSource:0}: Error finding container ec99ba591fb1b2e14ec194d93ee3f2a5adab090c2805cf90d748b9ec6c7cb3bd: Status 404 returned error can't find the container with id ec99ba591fb1b2e14ec194d93ee3f2a5adab090c2805cf90d748b9ec6c7cb3bd Jun 02 05:25:49.748785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.748760 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 05:25:49.755992 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.755804 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.783940 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.783902 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" event={"ID":"165b8d8bc20dd92d757a5571670d3d63","Type":"ContainerStarted","Data":"0b62453661fc1fb5546de8d5b896285c523211d74f71b04248608dfe79de7532"} Jun 02 05:25:49.784794 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.784775 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" event={"ID":"d8560f6e96c5dd0d18e85416cd362a94","Type":"ContainerStarted","Data":"ec99ba591fb1b2e14ec194d93ee3f2a5adab090c2805cf90d748b9ec6c7cb3bd"} Jun 02 05:25:49.856993 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:49.856951 2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-22.ec2.internal\" not found" Jun 02 05:25:49.908716 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.908686 2561 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 05:25:49.945587 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.945560 2561 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 05:25:49.957553 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.957531 2561 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.968891 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.968869 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 05:25:49.969614 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.969602 2561 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" Jun 02 05:25:49.977785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:49.977768 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 05:25:50.635043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.635009 2561 apiserver.go:52] "Watching apiserver" Jun 02 05:25:50.642752 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.642725 2561 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 05:25:50.643064 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.643041 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn","openshift-cluster-node-tuning-operator/tuned-x2wfr","openshift-image-registry/node-ca-5mg2p","openshift-multus/network-metrics-daemon-4fgvs","openshift-network-operator/iptables-alerter-xdg84","openshift-ovn-kubernetes/ovnkube-node-tz8dl","kube-system/konnectivity-agent-7727j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal","openshift-multus/multus-additional-cni-plugins-vq6j4","openshift-multus/multus-thns8","openshift-network-diagnostics/network-check-target-9vkts"] Jun 02 05:25:50.645665 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.645641 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.646912 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.646605 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.647772 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.647751 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.647874 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.647840 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:50.647934 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:50.647910 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:25:50.648411 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.648392 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 05:25:50.648513 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.648420 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 05:25:50.648513 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.648435 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 05:25:50.648878 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.648670 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5bhpr\"" Jun 02 05:25:50.648878 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.648841 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xx8qv\"" Jun 02 05:25:50.649044 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.648976 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 05:25:50.649265 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.649245 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 05:25:50.649954 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.649895 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 05:25:50.650058 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.650024 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 05:25:50.651077 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.650637 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 05:25:50.651077 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.650821 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gzs9r\"" Jun 02 05:25:50.652612 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.652038 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.652612 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.652206 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.653312 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.653286 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7727j" Jun 02 05:25:50.654860 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.654575 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zpsnn\"" Jun 02 05:25:50.654860 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.654598 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.654860 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.654674 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 05:25:50.654860 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.654674 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 05:25:50.655618 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.655582 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 05:25:50.655700 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.655648 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 05:25:50.655757 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.655737 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 05:25:50.655894 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.655651 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 05:25:50.656019 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.655930 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 05:25:50.656019 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.655988 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h9nml\"" Jun 02 05:25:50.656159 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.655885 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 05:25:50.656159 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.656108 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 05:25:50.656285 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.656234 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 05:25:50.656285 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.656269 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sg4gj\"" Jun 02 05:25:50.656473 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.656455 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.657142 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.657351 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.657439 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.657619 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.657648 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-v6g5t\"" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.657765 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.658850 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:50.658925 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:25:50.659542 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.658953 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-thns8" Jun 02 05:25:50.660765 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.660736 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 05:25:50.661144 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.661120 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nfzk4\"" Jun 02 05:25:50.667369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667344 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.667478 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667447 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysconfig\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.667533 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667485 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tp7v\" (UniqueName: \"kubernetes.io/projected/420349fd-ebb4-48ed-8c9f-3a60a886d66c-kube-api-access-8tp7v\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.667533 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-netns\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.667637 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667549 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/337004e8-eb54-4324-9126-4804d74d0dff-serviceca\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.667637 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667580 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46aef684-2246-40d2-ae8e-ad3f08626234-konnectivity-ca\") pod \"konnectivity-agent-7727j\" (UID: \"46aef684-2246-40d2-ae8e-ad3f08626234\") " pod="kube-system/konnectivity-agent-7727j" Jun 02 05:25:50.667637 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667612 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-sys\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.667775 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667642 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-lib-modules\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.667775 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667696 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64d2f732-0bdb-465f-836d-00274ec2d825-etc-tuned\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.667913 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghkpq\" (UniqueName: \"kubernetes.io/projected/9b89f038-8a67-4eca-bf3e-b36cc62d2165-kube-api-access-ghkpq\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.667994 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.667978 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-etc-selinux\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.668071 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668050 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrtxr\" (UniqueName: \"kubernetes.io/projected/a99b8d9f-7723-40bc-be05-b17d605888b4-kube-api-access-xrtxr\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:50.668139 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668092 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovnkube-config\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.668139 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668128 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-device-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.668227 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668156 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b89f038-8a67-4eca-bf3e-b36cc62d2165-host-slash\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.668227 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668202 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-system-cni-dir\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.668320 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668233 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.668320 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668260 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-878v5\" (UniqueName: \"kubernetes.io/projected/27ebee55-fb21-44db-8811-df38a448e458-kube-api-access-878v5\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.668320 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668288 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-cni-multus\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.668453 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668336 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-conf-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.668453 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668405 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64d2f732-0bdb-465f-836d-00274ec2d825-tmp\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.668453 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668441 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-system-cni-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.668565 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668480 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-env-overrides\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.668565 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668508 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46aef684-2246-40d2-ae8e-ad3f08626234-agent-certs\") pod \"konnectivity-agent-7727j\" (UID: \"46aef684-2246-40d2-ae8e-ad3f08626234\") " pod="kube-system/konnectivity-agent-7727j" Jun 02 05:25:50.668565 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668537 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-kubernetes\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.668717 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668565 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-var-lib-kubelet\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.668717 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668596 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cnibin\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.668717 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668622 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-multus-certs\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.668717 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668652 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-etc-kubernetes\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.668717 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668681 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-systemd\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.668880 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668746 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-etc-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.668880 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668793 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-cni-bin\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.668880 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668823 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysctl-conf\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.668880 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668846 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nsl9\" (UniqueName: \"kubernetes.io/projected/64d2f732-0bdb-465f-836d-00274ec2d825-kube-api-access-4nsl9\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.668880 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668875 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b89f038-8a67-4eca-bf3e-b36cc62d2165-iptables-alerter-script\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.669056 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668901 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-os-release\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.669056 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668919 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/337004e8-eb54-4324-9126-4804d74d0dff-host\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.669056 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.668992 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-ovn\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.669056 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669021 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-run-ovn-kubernetes\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.669056 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669052 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-run\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.669297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669078 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.669297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-cnibin\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.669297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvbjn\" (UniqueName: \"kubernetes.io/projected/337004e8-eb54-4324-9126-4804d74d0dff-kube-api-access-jvbjn\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.669297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669180 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-var-lib-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.669772 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669682 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-log-socket\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.669772 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669714 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-cni-netd\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.669772 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669745 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-systemd\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.669987 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sk7p\" (UniqueName: \"kubernetes.io/projected/e7aa2256-b292-46a9-8840-3ac46ae71b59-kube-api-access-7sk7p\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.669987 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669809 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-socket-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.669987 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669838 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovnkube-script-lib\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.669987 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669867 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-socket-dir-parent\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.669987 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669897 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-registration-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.669987 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669927 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-sys-fs\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.670282 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.669985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:50.670688 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670665 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqdrn\" (UniqueName: \"kubernetes.io/projected/71860b69-2110-4ccf-a8dc-55a86ce967a0-kube-api-access-kqdrn\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.670779 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670709 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-host\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.670779 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670735 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.670779 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670764 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-kubelet\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.670923 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670787 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.670923 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670808 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-node-log\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.670923 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670829 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.670923 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-os-release\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.670923 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670869 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7aa2256-b292-46a9-8840-3ac46ae71b59-cni-binary-copy\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.670923 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670890 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.670923 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670913 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-kubelet\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670929 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-systemd-units\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670943 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-slash\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.670976 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovn-node-metrics-cert\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.671211 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-k8s-cni-cncf-io\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.671243 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-daemon-config\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.671266 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-run-netns\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.671297 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysctl-d\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.671339 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-cni-bin\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.671357 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-modprobe-d\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.671369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.671370 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-cni-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.671670 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.671384 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-hostroot\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.758306 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.758278 2561 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 05:25:50.771819 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.771792 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xrtxr\" (UniqueName: \"kubernetes.io/projected/a99b8d9f-7723-40bc-be05-b17d605888b4-kube-api-access-xrtxr\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:50.771956 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.771831 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovnkube-config\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.771956 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.771877 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-device-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.771956 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.771916 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b89f038-8a67-4eca-bf3e-b36cc62d2165-host-slash\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.771956 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.771939 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-system-cni-dir\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.771984 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772014 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-878v5\" (UniqueName: \"kubernetes.io/projected/27ebee55-fb21-44db-8811-df38a448e458-kube-api-access-878v5\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772039 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-cni-multus\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772045 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b89f038-8a67-4eca-bf3e-b36cc62d2165-host-slash\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772060 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-conf-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772065 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-device-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772065 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-system-cni-dir\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772109 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-conf-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772113 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-cni-multus\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772187 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64d2f732-0bdb-465f-836d-00274ec2d825-tmp\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772222 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-system-cni-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772247 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-env-overrides\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772271 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46aef684-2246-40d2-ae8e-ad3f08626234-agent-certs\") pod \"konnectivity-agent-7727j\" (UID: \"46aef684-2246-40d2-ae8e-ad3f08626234\") " pod="kube-system/konnectivity-agent-7727j" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772298 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-kubernetes\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772323 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-var-lib-kubelet\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772347 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cnibin\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772329 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-system-cni-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772373 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-multus-certs\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772397 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-etc-kubernetes\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772412 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-kubernetes\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772421 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-systemd\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772445 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-etc-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772470 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-cni-bin\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772496 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysctl-conf\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772510 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772539 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nsl9\" (UniqueName: \"kubernetes.io/projected/64d2f732-0bdb-465f-836d-00274ec2d825-kube-api-access-4nsl9\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772568 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b89f038-8a67-4eca-bf3e-b36cc62d2165-iptables-alerter-script\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772571 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovnkube-config\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772596 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-os-release\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772599 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772638 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/337004e8-eb54-4324-9126-4804d74d0dff-host\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772660 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-os-release\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772665 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-ovn\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772708 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-ovn\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772709 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-etc-kubernetes\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772708 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-var-lib-kubelet\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772770 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-run-ovn-kubernetes\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772795 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-run\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772797 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-env-overrides\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772819 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.772858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772848 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-cnibin\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772874 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbjn\" (UniqueName: \"kubernetes.io/projected/337004e8-eb54-4324-9126-4804d74d0dff-kube-api-access-jvbjn\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772881 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-etc-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772899 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-var-lib-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772923 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-log-socket\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772944 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-cni-netd\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772985 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-systemd\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773009 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysctl-conf\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773010 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7sk7p\" (UniqueName: \"kubernetes.io/projected/e7aa2256-b292-46a9-8840-3ac46ae71b59-kube-api-access-7sk7p\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773048 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-socket-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773075 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovnkube-script-lib\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773123 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-socket-dir-parent\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773166 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-registration-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773192 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-sys-fs\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773218 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773242 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kqdrn\" (UniqueName: \"kubernetes.io/projected/71860b69-2110-4ccf-a8dc-55a86ce967a0-kube-api-access-kqdrn\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773265 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-host\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.773557 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773287 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773311 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-kubelet\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773342 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773370 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773377 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9b89f038-8a67-4eca-bf3e-b36cc62d2165-iptables-alerter-script\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773394 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-node-log\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772661 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-systemd\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773419 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773444 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-os-release\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773450 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cnibin\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773472 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7aa2256-b292-46a9-8840-3ac46ae71b59-cni-binary-copy\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773489 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-run-ovn-kubernetes\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773498 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773539 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-var-lib-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773559 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773568 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-kubelet\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773596 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-systemd-units\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774111 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773608 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-run\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773611 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773620 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-slash\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773651 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovn-node-metrics-cert\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773664 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-socket-dir-parent\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773677 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-k8s-cni-cncf-io\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773697 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-socket-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773702 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-daemon-config\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773705 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-registration-dir\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773736 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-run-netns\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.772848 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-cni-bin\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773804 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-systemd\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773849 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-os-release\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.773882 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-run-netns\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774199 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovnkube-script-lib\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774232 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-node-log\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774252 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-kubelet\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774272 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-log-socket\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.774620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774303 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-run-openvswitch\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774338 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-systemd-units\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774340 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-cni-netd\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774369 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-slash\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774418 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-k8s-cni-cncf-io\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774486 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-daemon-config\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774538 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-sys-fs\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:50.774605 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774642 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/420349fd-ebb4-48ed-8c9f-3a60a886d66c-cni-binary-copy\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774299 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7aa2256-b292-46a9-8840-3ac46ae71b59-cni-binary-copy\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774692 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-cnibin\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:50.774725 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs podName:a99b8d9f-7723-40bc-be05-b17d605888b4 nodeName:}" failed. No retries permitted until 2026-06-02 05:25:51.274656653 +0000 UTC m=+2.985105897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs") pod "network-metrics-daemon-4fgvs" (UID: "a99b8d9f-7723-40bc-be05-b17d605888b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774749 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-host\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774867 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-kubelet\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774867 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/420349fd-ebb4-48ed-8c9f-3a60a886d66c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774901 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysctl-d\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774907 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-multus-certs\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774931 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-cni-bin\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.775591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774934 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/337004e8-eb54-4324-9126-4804d74d0dff-host\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.774976 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-modprobe-d\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775012 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-var-lib-cni-bin\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775030 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-cni-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775041 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysctl-d\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775046 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-modprobe-d\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775081 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-hostroot\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775096 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-multus-cni-dir\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775114 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775131 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-hostroot\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775198 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysconfig\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775232 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71860b69-2110-4ccf-a8dc-55a86ce967a0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775277 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8tp7v\" (UniqueName: \"kubernetes.io/projected/420349fd-ebb4-48ed-8c9f-3a60a886d66c-kube-api-access-8tp7v\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775287 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-etc-sysconfig\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775334 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-netns\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775443 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/337004e8-eb54-4324-9126-4804d74d0dff-serviceca\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775494 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46aef684-2246-40d2-ae8e-ad3f08626234-konnectivity-ca\") pod \"konnectivity-agent-7727j\" (UID: \"46aef684-2246-40d2-ae8e-ad3f08626234\") " pod="kube-system/konnectivity-agent-7727j" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775519 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-sys\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.776506 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775562 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-lib-modules\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775585 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64d2f732-0bdb-465f-836d-00274ec2d825-etc-tuned\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775611 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ghkpq\" (UniqueName: \"kubernetes.io/projected/9b89f038-8a67-4eca-bf3e-b36cc62d2165-kube-api-access-ghkpq\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775637 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-etc-selinux\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775740 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/27ebee55-fb21-44db-8811-df38a448e458-etc-selinux\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775779 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-sys\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775813 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7aa2256-b292-46a9-8840-3ac46ae71b59-host-run-netns\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.775857 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/337004e8-eb54-4324-9126-4804d74d0dff-serviceca\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.776107 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64d2f732-0bdb-465f-836d-00274ec2d825-tmp\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.776214 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64d2f732-0bdb-465f-836d-00274ec2d825-lib-modules\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.776275 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46aef684-2246-40d2-ae8e-ad3f08626234-konnectivity-ca\") pod \"konnectivity-agent-7727j\" (UID: \"46aef684-2246-40d2-ae8e-ad3f08626234\") " pod="kube-system/konnectivity-agent-7727j" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.777192 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46aef684-2246-40d2-ae8e-ad3f08626234-agent-certs\") pod \"konnectivity-agent-7727j\" (UID: \"46aef684-2246-40d2-ae8e-ad3f08626234\") " pod="kube-system/konnectivity-agent-7727j" Jun 02 05:25:50.777315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.777193 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71860b69-2110-4ccf-a8dc-55a86ce967a0-ovn-node-metrics-cert\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.778414 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.778389 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64d2f732-0bdb-465f-836d-00274ec2d825-etc-tuned\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.781140 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.781119 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nsl9\" (UniqueName: \"kubernetes.io/projected/64d2f732-0bdb-465f-836d-00274ec2d825-kube-api-access-4nsl9\") pod \"tuned-x2wfr\" (UID: \"64d2f732-0bdb-465f-836d-00274ec2d825\") " pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.781495 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.781475 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-878v5\" (UniqueName: \"kubernetes.io/projected/27ebee55-fb21-44db-8811-df38a448e458-kube-api-access-878v5\") pod \"aws-ebs-csi-driver-node-dttvn\" (UID: \"27ebee55-fb21-44db-8811-df38a448e458\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.781576 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.781544 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrtxr\" (UniqueName: \"kubernetes.io/projected/a99b8d9f-7723-40bc-be05-b17d605888b4-kube-api-access-xrtxr\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:50.781790 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.781773 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sk7p\" (UniqueName: \"kubernetes.io/projected/e7aa2256-b292-46a9-8840-3ac46ae71b59-kube-api-access-7sk7p\") pod \"multus-thns8\" (UID: \"e7aa2256-b292-46a9-8840-3ac46ae71b59\") " pod="openshift-multus/multus-thns8" Jun 02 05:25:50.790042 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.790016 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvbjn\" (UniqueName: \"kubernetes.io/projected/337004e8-eb54-4324-9126-4804d74d0dff-kube-api-access-jvbjn\") pod \"node-ca-5mg2p\" (UID: \"337004e8-eb54-4324-9126-4804d74d0dff\") " pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.790131 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.790098 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqdrn\" (UniqueName: \"kubernetes.io/projected/71860b69-2110-4ccf-a8dc-55a86ce967a0-kube-api-access-kqdrn\") pod \"ovnkube-node-tz8dl\" (UID: \"71860b69-2110-4ccf-a8dc-55a86ce967a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.790179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.790148 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghkpq\" (UniqueName: \"kubernetes.io/projected/9b89f038-8a67-4eca-bf3e-b36cc62d2165-kube-api-access-ghkpq\") pod \"iptables-alerter-xdg84\" (UID: \"9b89f038-8a67-4eca-bf3e-b36cc62d2165\") " pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.792593 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.792570 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tp7v\" (UniqueName: \"kubernetes.io/projected/420349fd-ebb4-48ed-8c9f-3a60a886d66c-kube-api-access-8tp7v\") pod \"multus-additional-cni-plugins-vq6j4\" (UID: \"420349fd-ebb4-48ed-8c9f-3a60a886d66c\") " pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:50.876983 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.876930 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:50.883461 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:50.883431 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 05:25:50.883461 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:50.883459 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 05:25:50.883654 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:50.883473 2561 projected.go:194] Error preparing data for projected volume kube-api-access-p24nd for pod openshift-network-diagnostics/network-check-target-9vkts: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:50.883654 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:50.883573 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd podName:ddfa33ef-1107-42ae-b2aa-b281becf554b nodeName:}" failed. No retries permitted until 2026-06-02 05:25:51.383551833 +0000 UTC m=+3.094001074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p24nd" (UniqueName: "kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd") pod "network-check-target-9vkts" (UID: "ddfa33ef-1107-42ae-b2aa-b281becf554b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:50.958274 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.958166 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" Jun 02 05:25:50.966949 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.966925 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" Jun 02 05:25:50.976468 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.976448 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5mg2p" Jun 02 05:25:50.984120 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.984101 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xdg84" Jun 02 05:25:50.989690 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.989673 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:25:50.999322 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:50.999300 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7727j" Jun 02 05:25:51.005889 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.005863 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" Jun 02 05:25:51.011437 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.011417 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-thns8" Jun 02 05:25:51.028121 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.028086 2561 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 05:25:51.100129 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.100093 2561 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 05:25:51.279906 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.279826 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:51.280070 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:51.280001 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:51.280070 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:51.280056 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs podName:a99b8d9f-7723-40bc-be05-b17d605888b4 nodeName:}" failed. No retries permitted until 2026-06-02 05:25:52.280041395 +0000 UTC m=+3.990490632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs") pod "network-metrics-daemon-4fgvs" (UID: "a99b8d9f-7723-40bc-be05-b17d605888b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:51.387350 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:51.387319 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ebee55_fb21_44db_8811_df38a448e458.slice/crio-27f9527487ed5e3652d6fcd8b3e6d9543d624fcae908dbba5b928724a66de11a WatchSource:0}: Error finding container 27f9527487ed5e3652d6fcd8b3e6d9543d624fcae908dbba5b928724a66de11a: Status 404 returned error can't find the container with id 27f9527487ed5e3652d6fcd8b3e6d9543d624fcae908dbba5b928724a66de11a Jun 02 05:25:51.388381 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:51.388341 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337004e8_eb54_4324_9126_4804d74d0dff.slice/crio-15ceeaa22c79f1a6ff4d37ade5fcd210eca198b6ce5301e0e8239b73ea39a572 WatchSource:0}: Error finding container 15ceeaa22c79f1a6ff4d37ade5fcd210eca198b6ce5301e0e8239b73ea39a572: Status 404 returned error can't find the container with id 15ceeaa22c79f1a6ff4d37ade5fcd210eca198b6ce5301e0e8239b73ea39a572 Jun 02 05:25:51.392368 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:51.392244 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420349fd_ebb4_48ed_8c9f_3a60a886d66c.slice/crio-f1f30a64489b0f16e38d304b61af4523ca9cf8a16598f7efc64a0e7d7c5293cf WatchSource:0}: Error finding container f1f30a64489b0f16e38d304b61af4523ca9cf8a16598f7efc64a0e7d7c5293cf: Status 404 returned error can't find the container with id f1f30a64489b0f16e38d304b61af4523ca9cf8a16598f7efc64a0e7d7c5293cf Jun 02 05:25:51.394584 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:51.394537 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46aef684_2246_40d2_ae8e_ad3f08626234.slice/crio-b1bb035148ed0c79ed24fd76d38c0716ef196ab124036849c0fa221b10f44acf WatchSource:0}: Error finding container b1bb035148ed0c79ed24fd76d38c0716ef196ab124036849c0fa221b10f44acf: Status 404 returned error can't find the container with id b1bb035148ed0c79ed24fd76d38c0716ef196ab124036849c0fa221b10f44acf Jun 02 05:25:51.396253 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:51.396230 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71860b69_2110_4ccf_a8dc_55a86ce967a0.slice/crio-4f1c2b68156151bd36bb99d01325c02ed8d4142e076cd0321d224a3d89201ddb WatchSource:0}: Error finding container 4f1c2b68156151bd36bb99d01325c02ed8d4142e076cd0321d224a3d89201ddb: Status 404 returned error can't find the container with id 4f1c2b68156151bd36bb99d01325c02ed8d4142e076cd0321d224a3d89201ddb Jun 02 05:25:51.397218 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:51.396843 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64d2f732_0bdb_465f_836d_00274ec2d825.slice/crio-e04f74b0f9f58ca77d36ee6c0e3a3e7f25a5732c9268f4ad4ac8bb8d24324167 WatchSource:0}: Error finding container e04f74b0f9f58ca77d36ee6c0e3a3e7f25a5732c9268f4ad4ac8bb8d24324167: Status 404 returned error can't find the container with id e04f74b0f9f58ca77d36ee6c0e3a3e7f25a5732c9268f4ad4ac8bb8d24324167 Jun 02 05:25:51.398056 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:51.397546 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7aa2256_b292_46a9_8840_3ac46ae71b59.slice/crio-1a10924f6dc2fe51623157fe7f2d4aec5abb26b30d313ab5048d0d97ea7b181a WatchSource:0}: Error finding container 1a10924f6dc2fe51623157fe7f2d4aec5abb26b30d313ab5048d0d97ea7b181a: Status 404 returned error can't find the container with id 1a10924f6dc2fe51623157fe7f2d4aec5abb26b30d313ab5048d0d97ea7b181a Jun 02 05:25:51.398839 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:25:51.398787 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b89f038_8a67_4eca_bf3e_b36cc62d2165.slice/crio-15ff1a8a3976891acff555028d537863e7489caa36987c61c26ae64dce0d774d WatchSource:0}: Error finding container 15ff1a8a3976891acff555028d537863e7489caa36987c61c26ae64dce0d774d: Status 404 returned error can't find the container with id 15ff1a8a3976891acff555028d537863e7489caa36987c61c26ae64dce0d774d Jun 02 05:25:51.419049 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.419029 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mpm9c" Jun 02 05:25:51.427622 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.427603 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mpm9c" Jun 02 05:25:51.481249 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.481218 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:51.481405 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:51.481388 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 05:25:51.481497 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:51.481410 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 05:25:51.481497 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:51.481422 2561 projected.go:194] Error preparing data for projected volume kube-api-access-p24nd for pod openshift-network-diagnostics/network-check-target-9vkts: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:51.481497 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:51.481476 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd podName:ddfa33ef-1107-42ae-b2aa-b281becf554b nodeName:}" failed. No retries permitted until 2026-06-02 05:25:52.481457038 +0000 UTC m=+4.191906279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p24nd" (UniqueName: "kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd") pod "network-check-target-9vkts" (UID: "ddfa33ef-1107-42ae-b2aa-b281becf554b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:51.782683 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.782644 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:51.783038 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:51.782780 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:25:51.789662 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.789630 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xdg84" event={"ID":"9b89f038-8a67-4eca-bf3e-b36cc62d2165","Type":"ContainerStarted","Data":"15ff1a8a3976891acff555028d537863e7489caa36987c61c26ae64dce0d774d"} Jun 02 05:25:51.790887 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.790861 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-thns8" event={"ID":"e7aa2256-b292-46a9-8840-3ac46ae71b59","Type":"ContainerStarted","Data":"1a10924f6dc2fe51623157fe7f2d4aec5abb26b30d313ab5048d0d97ea7b181a"} Jun 02 05:25:51.792369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.792346 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" event={"ID":"64d2f732-0bdb-465f-836d-00274ec2d825","Type":"ContainerStarted","Data":"e04f74b0f9f58ca77d36ee6c0e3a3e7f25a5732c9268f4ad4ac8bb8d24324167"} Jun 02 05:25:51.793739 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.793715 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"4f1c2b68156151bd36bb99d01325c02ed8d4142e076cd0321d224a3d89201ddb"} Jun 02 05:25:51.796267 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.796124 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" event={"ID":"420349fd-ebb4-48ed-8c9f-3a60a886d66c","Type":"ContainerStarted","Data":"f1f30a64489b0f16e38d304b61af4523ca9cf8a16598f7efc64a0e7d7c5293cf"} Jun 02 05:25:51.797670 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.797641 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" event={"ID":"27ebee55-fb21-44db-8811-df38a448e458","Type":"ContainerStarted","Data":"27f9527487ed5e3652d6fcd8b3e6d9543d624fcae908dbba5b928724a66de11a"} Jun 02 05:25:51.802077 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.801870 2561 generic.go:358] "Generic (PLEG): container finished" podID="d8560f6e96c5dd0d18e85416cd362a94" containerID="d699123fcf09dffc86d20e311a3666a4b63edb549a3dee1305f68817c746a089" exitCode=0 Jun 02 05:25:51.802077 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.801934 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" event={"ID":"d8560f6e96c5dd0d18e85416cd362a94","Type":"ContainerDied","Data":"d699123fcf09dffc86d20e311a3666a4b63edb549a3dee1305f68817c746a089"} Jun 02 05:25:51.810397 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.810271 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" event={"ID":"165b8d8bc20dd92d757a5571670d3d63","Type":"ContainerStarted","Data":"6a6d3a745b91b5cafebef6b9e6fff265b9c37e7fbdaeeb10bc9650d1418388d8"} Jun 02 05:25:51.813862 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.813824 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7727j" event={"ID":"46aef684-2246-40d2-ae8e-ad3f08626234","Type":"ContainerStarted","Data":"b1bb035148ed0c79ed24fd76d38c0716ef196ab124036849c0fa221b10f44acf"} Jun 02 05:25:51.817024 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:51.816258 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5mg2p" event={"ID":"337004e8-eb54-4324-9126-4804d74d0dff","Type":"ContainerStarted","Data":"15ceeaa22c79f1a6ff4d37ade5fcd210eca198b6ce5301e0e8239b73ea39a572"} Jun 02 05:25:52.288756 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:52.288722 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:52.288927 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:52.288906 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:52.289035 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:52.288998 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs podName:a99b8d9f-7723-40bc-be05-b17d605888b4 nodeName:}" failed. No retries permitted until 2026-06-02 05:25:54.288978381 +0000 UTC m=+5.999427619 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs") pod "network-metrics-daemon-4fgvs" (UID: "a99b8d9f-7723-40bc-be05-b17d605888b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:52.429067 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:52.429026 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 05:20:51 +0000 UTC" deadline="2028-03-20 05:10:34.771573576 +0000 UTC" Jun 02 05:25:52.429067 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:52.429065 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15767h44m42.34251191s" Jun 02 05:25:52.490064 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:52.490018 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:52.490227 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:52.490196 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 05:25:52.490227 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:52.490219 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 05:25:52.490345 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:52.490234 2561 projected.go:194] Error preparing data for projected volume kube-api-access-p24nd for pod openshift-network-diagnostics/network-check-target-9vkts: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:52.490345 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:52.490298 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd podName:ddfa33ef-1107-42ae-b2aa-b281becf554b nodeName:}" failed. No retries permitted until 2026-06-02 05:25:54.490278011 +0000 UTC m=+6.200727255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p24nd" (UniqueName: "kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd") pod "network-check-target-9vkts" (UID: "ddfa33ef-1107-42ae-b2aa-b281becf554b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:52.783131 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:52.782868 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:52.783131 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:52.783031 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:25:52.832301 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:52.832251 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" event={"ID":"d8560f6e96c5dd0d18e85416cd362a94","Type":"ContainerStarted","Data":"4aaed34c3a13cca049fd49b980314a5347ec1992859ed24bf1ed51ba1994667b"} Jun 02 05:25:52.849843 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:52.849383 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-22.ec2.internal" podStartSLOduration=3.8493661980000002 podStartE2EDuration="3.849366198s" podCreationTimestamp="2026-06-02 05:25:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:25:52.848897931 +0000 UTC m=+4.559347178" watchObservedRunningTime="2026-06-02 05:25:52.849366198 +0000 UTC m=+4.559815446" Jun 02 05:25:52.849843 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:52.849731 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-22.ec2.internal" podStartSLOduration=3.8497219400000002 podStartE2EDuration="3.84972194s" podCreationTimestamp="2026-06-02 05:25:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:25:51.836893263 +0000 UTC m=+3.547342509" watchObservedRunningTime="2026-06-02 05:25:52.84972194 +0000 UTC m=+4.560171186" Jun 02 05:25:53.430182 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:53.430135 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 05:20:51 +0000 UTC" deadline="2028-03-11 11:54:18.259738924 +0000 UTC" Jun 02 05:25:53.430182 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:53.430179 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15558h28m24.829563046s" Jun 02 05:25:53.781805 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:53.781724 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:53.781956 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:53.781853 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:25:54.305828 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:54.305786 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:54.306275 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:54.305943 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:54.306275 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:54.306034 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs podName:a99b8d9f-7723-40bc-be05-b17d605888b4 nodeName:}" failed. No retries permitted until 2026-06-02 05:25:58.306007057 +0000 UTC m=+10.016456294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs") pod "network-metrics-daemon-4fgvs" (UID: "a99b8d9f-7723-40bc-be05-b17d605888b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:54.507250 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:54.507212 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:54.507397 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:54.507378 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 05:25:54.507397 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:54.507397 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 05:25:54.507500 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:54.507411 2561 projected.go:194] Error preparing data for projected volume kube-api-access-p24nd for pod openshift-network-diagnostics/network-check-target-9vkts: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:54.507500 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:54.507472 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd podName:ddfa33ef-1107-42ae-b2aa-b281becf554b nodeName:}" failed. No retries permitted until 2026-06-02 05:25:58.507452472 +0000 UTC m=+10.217901711 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p24nd" (UniqueName: "kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd") pod "network-check-target-9vkts" (UID: "ddfa33ef-1107-42ae-b2aa-b281becf554b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:54.783169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:54.783094 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:54.783323 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:54.783226 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:25:55.782758 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:55.782224 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:55.782758 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:55.782369 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:25:56.782680 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:56.782638 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:56.782851 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:56.782786 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:25:57.781990 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:57.781932 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:57.782153 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:57.782092 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:25:58.337647 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:58.337599 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:58.338086 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:58.337779 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:58.338086 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:58.337857 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs podName:a99b8d9f-7723-40bc-be05-b17d605888b4 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:06.337835652 +0000 UTC m=+18.048284901 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs") pod "network-metrics-daemon-4fgvs" (UID: "a99b8d9f-7723-40bc-be05-b17d605888b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:25:58.539637 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:58.539594 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:58.539809 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:58.539729 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 05:25:58.539809 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:58.539747 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 05:25:58.539809 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:58.539759 2561 projected.go:194] Error preparing data for projected volume kube-api-access-p24nd for pod openshift-network-diagnostics/network-check-target-9vkts: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:58.540099 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:58.539812 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd podName:ddfa33ef-1107-42ae-b2aa-b281becf554b nodeName:}" failed. No retries permitted until 2026-06-02 05:26:06.539798633 +0000 UTC m=+18.250247857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p24nd" (UniqueName: "kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd") pod "network-check-target-9vkts" (UID: "ddfa33ef-1107-42ae-b2aa-b281becf554b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:25:58.783136 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:58.783057 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:25:58.783296 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:58.783183 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:25:59.781952 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:25:59.781919 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:25:59.782462 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:25:59.782088 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:00.782827 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:00.782789 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:00.783306 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:00.782925 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:01.782112 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:01.782081 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:01.782263 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:01.782182 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:01.869153 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:01.869115 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8qmnz"] Jun 02 05:26:01.885871 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:01.885844 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:01.885952 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:01.885930 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:01.967287 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:01.967249 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47e6d278-6f80-4c82-857d-564ed5515845-kubelet-config\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:01.967421 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:01.967334 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:01.967421 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:01.967352 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47e6d278-6f80-4c82-857d-564ed5515845-dbus\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:02.068455 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:02.068419 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47e6d278-6f80-4c82-857d-564ed5515845-kubelet-config\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:02.068598 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:02.068506 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:02.068598 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:02.068530 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47e6d278-6f80-4c82-857d-564ed5515845-dbus\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:02.068598 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:02.068563 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47e6d278-6f80-4c82-857d-564ed5515845-kubelet-config\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:02.068702 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:02.068609 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47e6d278-6f80-4c82-857d-564ed5515845-dbus\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:02.068702 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:02.068647 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:02.068760 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:02.068703 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret podName:47e6d278-6f80-4c82-857d-564ed5515845 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:02.568687388 +0000 UTC m=+14.279136628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret") pod "global-pull-secret-syncer-8qmnz" (UID: "47e6d278-6f80-4c82-857d-564ed5515845") : object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:02.573228 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:02.573190 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:02.573405 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:02.573314 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:02.573405 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:02.573372 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret podName:47e6d278-6f80-4c82-857d-564ed5515845 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:03.57335638 +0000 UTC m=+15.283805605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret") pod "global-pull-secret-syncer-8qmnz" (UID: "47e6d278-6f80-4c82-857d-564ed5515845") : object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:02.782423 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:02.782387 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:02.782598 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:02.782543 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:03.579870 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.579835 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:03.580316 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:03.580025 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:03.580316 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:03.580094 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret podName:47e6d278-6f80-4c82-857d-564ed5515845 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:05.580075774 +0000 UTC m=+17.290525007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret") pod "global-pull-secret-syncer-8qmnz" (UID: "47e6d278-6f80-4c82-857d-564ed5515845") : object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:03.699082 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.699044 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-5bs54"] Jun 02 05:26:03.719426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.719387 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.722113 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.722086 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 05:26:03.722113 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.722109 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k4zcz\"" Jun 02 05:26:03.722317 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.722139 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 05:26:03.781107 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.781072 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdk7b\" (UniqueName: \"kubernetes.io/projected/ecb2d002-0193-47b8-a3a5-c896da8ba212-kube-api-access-cdk7b\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.781253 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.781119 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecb2d002-0193-47b8-a3a5-c896da8ba212-tmp-dir\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.781253 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.781207 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecb2d002-0193-47b8-a3a5-c896da8ba212-hosts-file\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.782099 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.782074 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:03.782199 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.782074 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:03.782239 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:03.782195 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:03.782279 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:03.782257 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:03.882453 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.882355 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecb2d002-0193-47b8-a3a5-c896da8ba212-hosts-file\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.882453 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.882423 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cdk7b\" (UniqueName: \"kubernetes.io/projected/ecb2d002-0193-47b8-a3a5-c896da8ba212-kube-api-access-cdk7b\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.882685 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.882462 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecb2d002-0193-47b8-a3a5-c896da8ba212-tmp-dir\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.882685 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.882515 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecb2d002-0193-47b8-a3a5-c896da8ba212-hosts-file\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.883105 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.883078 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecb2d002-0193-47b8-a3a5-c896da8ba212-tmp-dir\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:03.895692 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:03.895561 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdk7b\" (UniqueName: \"kubernetes.io/projected/ecb2d002-0193-47b8-a3a5-c896da8ba212-kube-api-access-cdk7b\") pod \"node-resolver-5bs54\" (UID: \"ecb2d002-0193-47b8-a3a5-c896da8ba212\") " pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:04.029261 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:04.029223 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5bs54" Jun 02 05:26:04.782609 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:04.782572 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:04.783074 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:04.782708 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:05.592920 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:05.592889 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:05.593052 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:05.593008 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:05.593089 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:05.593067 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret podName:47e6d278-6f80-4c82-857d-564ed5515845 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:09.593053432 +0000 UTC m=+21.303502656 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret") pod "global-pull-secret-syncer-8qmnz" (UID: "47e6d278-6f80-4c82-857d-564ed5515845") : object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:05.782268 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:05.782226 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:05.782458 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:05.782222 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:05.782458 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:05.782380 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:05.782458 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:05.782449 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:06.398597 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:06.398554 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:06.399112 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:06.398682 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:26:06.399112 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:06.398753 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs podName:a99b8d9f-7723-40bc-be05-b17d605888b4 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:22.398736034 +0000 UTC m=+34.109185258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs") pod "network-metrics-daemon-4fgvs" (UID: "a99b8d9f-7723-40bc-be05-b17d605888b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:26:06.600207 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:06.600162 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:06.600398 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:06.600361 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 05:26:06.600398 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:06.600389 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 05:26:06.600491 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:06.600403 2561 projected.go:194] Error preparing data for projected volume kube-api-access-p24nd for pod openshift-network-diagnostics/network-check-target-9vkts: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:26:06.600491 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:06.600463 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd podName:ddfa33ef-1107-42ae-b2aa-b281becf554b nodeName:}" failed. No retries permitted until 2026-06-02 05:26:22.600444337 +0000 UTC m=+34.310893585 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p24nd" (UniqueName: "kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd") pod "network-check-target-9vkts" (UID: "ddfa33ef-1107-42ae-b2aa-b281becf554b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:26:06.782655 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:06.782568 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:06.782800 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:06.782696 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:07.781882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:07.781845 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:07.782371 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:07.781845 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:07.782371 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:07.782003 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:07.782371 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:07.782083 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:08.783130 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:08.783093 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:08.783514 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:08.783201 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:09.619517 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:09.619468 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:09.619682 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:09.619638 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:09.619724 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:09.619702 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret podName:47e6d278-6f80-4c82-857d-564ed5515845 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:17.619687986 +0000 UTC m=+29.330137218 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret") pod "global-pull-secret-syncer-8qmnz" (UID: "47e6d278-6f80-4c82-857d-564ed5515845") : object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:09.782311 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:09.782278 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:09.782483 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:09.782288 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:09.782483 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:09.782395 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:09.782483 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:09.782461 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:10.388956 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:10.388925 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb2d002_0193_47b8_a3a5_c896da8ba212.slice/crio-e8d1c44358ad4f02dd9d87ed4d795de480457bead6b0c9d8c6ebb40847657e76 WatchSource:0}: Error finding container e8d1c44358ad4f02dd9d87ed4d795de480457bead6b0c9d8c6ebb40847657e76: Status 404 returned error can't find the container with id e8d1c44358ad4f02dd9d87ed4d795de480457bead6b0c9d8c6ebb40847657e76 Jun 02 05:26:10.782661 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.782510 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:10.782793 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:10.782757 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:10.865051 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.864998 2561 generic.go:358] "Generic (PLEG): container finished" podID="420349fd-ebb4-48ed-8c9f-3a60a886d66c" containerID="e8897b61d657388ce9d0bf7c7f5dd7092ee804108388a8b7c0c1ce7dfd49b3a1" exitCode=0 Jun 02 05:26:10.865154 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.865078 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" event={"ID":"420349fd-ebb4-48ed-8c9f-3a60a886d66c","Type":"ContainerDied","Data":"e8897b61d657388ce9d0bf7c7f5dd7092ee804108388a8b7c0c1ce7dfd49b3a1"} Jun 02 05:26:10.866472 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.866448 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" event={"ID":"27ebee55-fb21-44db-8811-df38a448e458","Type":"ContainerStarted","Data":"e0d87fb86b184101b8f0b3f4868dcce1514ab3bcfd816670cf0b08541e90e7a8"} Jun 02 05:26:10.867651 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.867624 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7727j" event={"ID":"46aef684-2246-40d2-ae8e-ad3f08626234","Type":"ContainerStarted","Data":"39294cbafe03e3f1cd767ead1a017d7d56ecc297e1b2d6c310d04193fbb35137"} Jun 02 05:26:10.868807 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.868789 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5mg2p" event={"ID":"337004e8-eb54-4324-9126-4804d74d0dff","Type":"ContainerStarted","Data":"56f6418272fa8d57f1c4e030920543fb6b7eeba571538dc5681d690e42a36768"} Jun 02 05:26:10.869977 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.869945 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5bs54" event={"ID":"ecb2d002-0193-47b8-a3a5-c896da8ba212","Type":"ContainerStarted","Data":"55376fac266297aed9176fba523044ce12859dc9b47cee5596190edaf7aadddb"} Jun 02 05:26:10.870077 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.869997 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5bs54" event={"ID":"ecb2d002-0193-47b8-a3a5-c896da8ba212","Type":"ContainerStarted","Data":"e8d1c44358ad4f02dd9d87ed4d795de480457bead6b0c9d8c6ebb40847657e76"} Jun 02 05:26:10.871394 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.871249 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-thns8" event={"ID":"e7aa2256-b292-46a9-8840-3ac46ae71b59","Type":"ContainerStarted","Data":"5e718fcb04c68f129c0f72c93441d00aa3c129943328131fa7ad8e1b16389101"} Jun 02 05:26:10.872492 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.872469 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" event={"ID":"64d2f732-0bdb-465f-836d-00274ec2d825","Type":"ContainerStarted","Data":"cb7c147519e0038faadb439def602c3bc52b5b0bd456a8de14d934a5283fbd32"} Jun 02 05:26:10.874109 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.874087 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"164aa0ee76db566b84efb6a3501b01a13d8cced3874ae015f559f0e0968fa363"} Jun 02 05:26:10.874191 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.874113 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"c24957c21e3406acbc5eb3cab89862d0204500c226426379d54c206740483164"} Jun 02 05:26:10.874191 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.874122 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"d117ba143c20bf4d3e349cd75ac766c58ad79ea1d679196f2607bb965769de76"} Jun 02 05:26:10.929495 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.929456 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5mg2p" podStartSLOduration=3.97769298 podStartE2EDuration="22.929444764s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:25:51.39007023 +0000 UTC m=+3.100519475" lastFinishedPulling="2026-06-02 05:26:10.341822022 +0000 UTC m=+22.052271259" observedRunningTime="2026-06-02 05:26:10.913055299 +0000 UTC m=+22.623504545" watchObservedRunningTime="2026-06-02 05:26:10.929444764 +0000 UTC m=+22.639894065" Jun 02 05:26:10.929749 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.929723 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7727j" podStartSLOduration=3.984356071 podStartE2EDuration="22.929715042s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:25:51.396457612 +0000 UTC m=+3.106906845" lastFinishedPulling="2026-06-02 05:26:10.341816579 +0000 UTC m=+22.052265816" observedRunningTime="2026-06-02 05:26:10.929179316 +0000 UTC m=+22.639628563" watchObservedRunningTime="2026-06-02 05:26:10.929715042 +0000 UTC m=+22.640164288" Jun 02 05:26:10.950982 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.950927 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-thns8" podStartSLOduration=3.96319114 podStartE2EDuration="22.950914758s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:25:51.399290992 +0000 UTC m=+3.109740215" lastFinishedPulling="2026-06-02 05:26:10.387014592 +0000 UTC m=+22.097463833" observedRunningTime="2026-06-02 05:26:10.950506684 +0000 UTC m=+22.660955924" watchObservedRunningTime="2026-06-02 05:26:10.950914758 +0000 UTC m=+22.661364005" Jun 02 05:26:10.970075 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:10.970038 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x2wfr" podStartSLOduration=3.98172805 podStartE2EDuration="22.97002578s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:25:51.399071455 +0000 UTC m=+3.109520695" lastFinishedPulling="2026-06-02 05:26:10.387369196 +0000 UTC m=+22.097818425" observedRunningTime="2026-06-02 05:26:10.969790111 +0000 UTC m=+22.680239358" watchObservedRunningTime="2026-06-02 05:26:10.97002578 +0000 UTC m=+22.680475025" Jun 02 05:26:11.784208 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:11.782198 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:11.784208 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:11.782557 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:11.784208 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:11.783008 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:11.784208 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:11.783131 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:11.823211 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:11.823183 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 05:26:11.879410 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:11.879359 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"c6fd75712cb1f2fc5c845d6d2459e1e0399ce090b5788fd2e541e353a7798c31"} Jun 02 05:26:11.879552 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:11.879433 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"ff68dbe9d34861b808584a28034e8b8cf98d9959b5b72802de980ece4bc35ae7"} Jun 02 05:26:11.879552 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:11.879446 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"e7488f26f39b56f5b5514b84dfbf77ed29b9ede5ac86de7a5c872a43060d3550"} Jun 02 05:26:11.880995 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:11.880951 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" event={"ID":"27ebee55-fb21-44db-8811-df38a448e458","Type":"ContainerStarted","Data":"3d9f5c77754827d34c677d0553c2a9e00d5ff047bf390be2b70748485acca98f"} Jun 02 05:26:12.359869 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.359796 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7727j" Jun 02 05:26:12.360523 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.360507 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7727j" Jun 02 05:26:12.384726 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.384678 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-5bs54" podStartSLOduration=9.384666738 podStartE2EDuration="9.384666738s" podCreationTimestamp="2026-06-02 05:26:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:26:10.986094667 +0000 UTC m=+22.696543912" watchObservedRunningTime="2026-06-02 05:26:12.384666738 +0000 UTC m=+24.095115981" Jun 02 05:26:12.736804 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.736636 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T05:26:11.823207254Z","UUID":"6052c3f3-6518-42d7-9dfa-4f68d7a556f5","Handler":null,"Name":"","Endpoint":""} Jun 02 05:26:12.740222 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.740195 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 05:26:12.740222 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.740229 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 05:26:12.782440 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.782409 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:12.782585 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:12.782545 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:12.883585 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.883554 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xdg84" event={"ID":"9b89f038-8a67-4eca-bf3e-b36cc62d2165","Type":"ContainerStarted","Data":"39aef5c39fb89329908ac1edc97dc279385080cb1d0af01b85b87c5341589dd5"} Jun 02 05:26:12.900101 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:12.900051 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xdg84" podStartSLOduration=5.913047911 podStartE2EDuration="24.9000338s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:25:51.401114777 +0000 UTC m=+3.111564001" lastFinishedPulling="2026-06-02 05:26:10.388100653 +0000 UTC m=+22.098549890" observedRunningTime="2026-06-02 05:26:12.899978486 +0000 UTC m=+24.610427733" watchObservedRunningTime="2026-06-02 05:26:12.9000338 +0000 UTC m=+24.610483049" Jun 02 05:26:13.782115 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:13.782084 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:13.782282 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:13.782094 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:13.782282 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:13.782192 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:13.782282 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:13.782262 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:13.888947 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:13.888912 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"c47b9869ec207a62ef44f117fe5f58bf5504606acb3ae12ebff78501c72eb9e5"} Jun 02 05:26:13.891304 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:13.891280 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" event={"ID":"27ebee55-fb21-44db-8811-df38a448e458","Type":"ContainerStarted","Data":"f47873df025ca58a1518554b139ea06b6b3e953e92481d8eaba7c31ee0ecdb1c"} Jun 02 05:26:13.891413 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:13.891392 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 02 05:26:14.495147 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:14.495112 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7727j" Jun 02 05:26:14.495737 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:14.495715 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7727j" Jun 02 05:26:14.513315 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:14.513256 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dttvn" podStartSLOduration=4.779911009 podStartE2EDuration="26.513240126s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:25:51.390077264 +0000 UTC m=+3.100526501" lastFinishedPulling="2026-06-02 05:26:13.123406375 +0000 UTC m=+24.833855618" observedRunningTime="2026-06-02 05:26:13.913801728 +0000 UTC m=+25.624250976" watchObservedRunningTime="2026-06-02 05:26:14.513240126 +0000 UTC m=+26.223689391" Jun 02 05:26:14.782120 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:14.782047 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:14.782270 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:14.782176 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:15.782328 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:15.782244 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:15.782328 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:15.782288 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:15.782874 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:15.782387 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:15.782874 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:15.782530 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:16.782781 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:16.782571 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:16.783140 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:16.782895 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:16.898476 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:16.898439 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" event={"ID":"71860b69-2110-4ccf-a8dc-55a86ce967a0","Type":"ContainerStarted","Data":"fba7a52675684cf6e90483efa6ccde1075b1c57d163401eccfb9a35b2007d0ef"} Jun 02 05:26:16.898786 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:16.898761 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:26:16.898883 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:16.898789 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:26:16.900242 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:16.900218 2561 generic.go:358] "Generic (PLEG): container finished" podID="420349fd-ebb4-48ed-8c9f-3a60a886d66c" containerID="8ca6ca6f1dcc380bb78295a94468c7910795a35e357dbf4a68a3f064666881cf" exitCode=0 Jun 02 05:26:16.900360 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:16.900258 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" event={"ID":"420349fd-ebb4-48ed-8c9f-3a60a886d66c","Type":"ContainerDied","Data":"8ca6ca6f1dcc380bb78295a94468c7910795a35e357dbf4a68a3f064666881cf"} Jun 02 05:26:16.915035 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:16.915017 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:26:16.956696 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:16.956649 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" podStartSLOduration=9.862191206 podStartE2EDuration="28.956635788s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:25:51.398015315 +0000 UTC m=+3.108464556" lastFinishedPulling="2026-06-02 05:26:10.492459906 +0000 UTC m=+22.202909138" observedRunningTime="2026-06-02 05:26:16.956081333 +0000 UTC m=+28.666530579" watchObservedRunningTime="2026-06-02 05:26:16.956635788 +0000 UTC m=+28.667085033" Jun 02 05:26:17.686774 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:17.686747 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:17.686892 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:17.686841 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:17.686892 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:17.686885 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret podName:47e6d278-6f80-4c82-857d-564ed5515845 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:33.686873417 +0000 UTC m=+45.397322641 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret") pod "global-pull-secret-syncer-8qmnz" (UID: "47e6d278-6f80-4c82-857d-564ed5515845") : object "kube-system"/"original-pull-secret" not registered Jun 02 05:26:17.782305 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:17.782279 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:17.782305 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:17.782312 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:17.782475 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:17.782379 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:17.782475 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:17.782424 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:17.904512 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:17.904304 2561 generic.go:358] "Generic (PLEG): container finished" podID="420349fd-ebb4-48ed-8c9f-3a60a886d66c" containerID="db76471894191c062341f0fedd53d743f81bf8dca8079d408abf34855f398d1a" exitCode=0 Jun 02 05:26:17.904512 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:17.904385 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" event={"ID":"420349fd-ebb4-48ed-8c9f-3a60a886d66c","Type":"ContainerDied","Data":"db76471894191c062341f0fedd53d743f81bf8dca8079d408abf34855f398d1a"} Jun 02 05:26:17.905190 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:17.905118 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:26:17.920809 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:17.920787 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:26:18.257665 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:18.257590 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9vkts"] Jun 02 05:26:18.257831 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:18.257685 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:18.257831 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:18.257762 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:18.263102 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:18.263077 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8qmnz"] Jun 02 05:26:18.263229 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:18.263168 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:18.263290 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:18.263269 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:18.263977 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:18.263927 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4fgvs"] Jun 02 05:26:18.264152 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:18.264124 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:18.264281 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:18.264256 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:18.908627 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:18.908547 2561 generic.go:358] "Generic (PLEG): container finished" podID="420349fd-ebb4-48ed-8c9f-3a60a886d66c" containerID="fe3592d94ea52360127e86d584f2293c8f5aac13b9b1cff6793e4b69ed4f0420" exitCode=0 Jun 02 05:26:18.909207 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:18.908624 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" event={"ID":"420349fd-ebb4-48ed-8c9f-3a60a886d66c","Type":"ContainerDied","Data":"fe3592d94ea52360127e86d584f2293c8f5aac13b9b1cff6793e4b69ed4f0420"} Jun 02 05:26:19.782294 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:19.782212 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:19.782443 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:19.782224 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:19.782443 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:19.782339 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:19.782443 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:19.782335 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:19.782585 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:19.782435 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:19.782585 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:19.782537 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:21.782650 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:21.782557 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:21.783185 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:21.782687 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:21.783185 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:21.782693 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qmnz" podUID="47e6d278-6f80-4c82-857d-564ed5515845" Jun 02 05:26:21.783185 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:21.782834 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fgvs" podUID="a99b8d9f-7723-40bc-be05-b17d605888b4" Jun 02 05:26:21.783185 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:21.782870 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:21.783185 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:21.782943 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vkts" podUID="ddfa33ef-1107-42ae-b2aa-b281becf554b" Jun 02 05:26:22.199330 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:22.199305 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5bs54_ecb2d002-0193-47b8-a3a5-c896da8ba212/dns-node-resolver/0.log" Jun 02 05:26:22.418220 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:22.418185 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:22.418400 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:22.418332 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:26:22.418400 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:22.418400 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs podName:a99b8d9f-7723-40bc-be05-b17d605888b4 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:54.418384747 +0000 UTC m=+66.128833987 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs") pod "network-metrics-daemon-4fgvs" (UID: "a99b8d9f-7723-40bc-be05-b17d605888b4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 05:26:22.619482 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:22.619441 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:22.619701 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:22.619632 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 05:26:22.619701 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:22.619665 2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 05:26:22.619701 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:22.619678 2561 projected.go:194] Error preparing data for projected volume kube-api-access-p24nd for pod openshift-network-diagnostics/network-check-target-9vkts: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:26:22.619857 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:22.619750 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd podName:ddfa33ef-1107-42ae-b2aa-b281becf554b nodeName:}" failed. No retries permitted until 2026-06-02 05:26:54.619729726 +0000 UTC m=+66.330178967 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p24nd" (UniqueName: "kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd") pod "network-check-target-9vkts" (UID: "ddfa33ef-1107-42ae-b2aa-b281becf554b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 05:26:23.156144 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.156109 2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-22.ec2.internal" event="NodeReady" Jun 02 05:26:23.156632 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.156256 2561 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 05:26:23.164875 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.164815 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5mg2p_337004e8-eb54-4324-9126-4804d74d0dff/node-ca/0.log" Jun 02 05:26:23.197216 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.197184 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c6d6cc778-kcxhm"] Jun 02 05:26:23.214312 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.214285 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kc7f9"] Jun 02 05:26:23.214492 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.214463 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.217893 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.217867 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 05:26:23.218236 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.218214 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bkp9z\"" Jun 02 05:26:23.218359 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.218337 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 05:26:23.218572 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.218531 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 05:26:23.222058 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.222037 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 05:26:23.225290 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.224385 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rw625"] Jun 02 05:26:23.236795 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.236636 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c6d6cc778-kcxhm"] Jun 02 05:26:23.236795 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.236662 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kc7f9"] Jun 02 05:26:23.236795 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.236699 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rw625"] Jun 02 05:26:23.236795 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.236754 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.236795 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.236769 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:23.239716 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.239309 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 05:26:23.239716 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.239331 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 05:26:23.239716 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.239369 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 05:26:23.239716 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.239452 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9f64n\"" Jun 02 05:26:23.240047 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.239754 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 05:26:23.240047 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.239816 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 05:26:23.240047 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.240041 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gnx8d\"" Jun 02 05:26:23.324374 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324336 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-tmp-dir\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.324589 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-bound-sa-token\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.324589 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324421 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mtj4\" (UniqueName: \"kubernetes.io/projected/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-kube-api-access-9mtj4\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.324589 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-installation-pull-secrets\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.324589 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324488 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqjrx\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-kube-api-access-kqjrx\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.324589 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324515 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aca9032d-5d74-4374-a10f-694d6ced4025-ca-trust-extracted\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.324790 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324590 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-config-volume\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.324790 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324635 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slqjb\" (UniqueName: \"kubernetes.io/projected/7c31483d-0694-44d8-9125-f41ddf67e3d9-kube-api-access-slqjb\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:23.324790 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324739 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-image-registry-private-configuration\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.324790 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324772 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.324953 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324809 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:23.324953 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324842 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-registry-certificates\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.324953 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324890 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-trusted-ca\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.324953 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.324921 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.337152 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.337123 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-j8zz4"] Jun 02 05:26:23.352304 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.352278 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-j8zz4"] Jun 02 05:26:23.352444 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.352403 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.355362 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.355337 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 02 05:26:23.355362 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.355359 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4bgd4\"" Jun 02 05:26:23.355511 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.355346 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 02 05:26:23.355511 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.355398 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 02 05:26:23.355719 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.355700 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 02 05:26:23.425879 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.425809 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjrx\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-kube-api-access-kqjrx\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.425879 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.425843 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aca9032d-5d74-4374-a10f-694d6ced4025-ca-trust-extracted\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426107 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.425998 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-config-volume\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.426107 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426053 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-slqjb\" (UniqueName: \"kubernetes.io/projected/7c31483d-0694-44d8-9125-f41ddf67e3d9-kube-api-access-slqjb\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:23.426209 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426128 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-image-registry-private-configuration\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426209 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426158 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426209 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/16712eb9-39a2-48fe-aa0c-787be9e0577f-signing-key\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.426319 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426216 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:23.426319 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426233 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fm5b\" (UniqueName: \"kubernetes.io/projected/16712eb9-39a2-48fe-aa0c-787be9e0577f-kube-api-access-5fm5b\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.426319 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426241 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aca9032d-5d74-4374-a10f-694d6ced4025-ca-trust-extracted\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426319 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426257 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-registry-certificates\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426319 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426295 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-trusted-ca\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426501 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426321 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.426501 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426393 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-tmp-dir\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.426501 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426422 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/16712eb9-39a2-48fe-aa0c-787be9e0577f-signing-cabundle\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.426501 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426451 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-bound-sa-token\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426501 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.426455 2561 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 05:26:23.426501 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.426470 2561 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6d6cc778-kcxhm: secret "image-registry-tls" not found Jun 02 05:26:23.426501 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426477 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9mtj4\" (UniqueName: \"kubernetes.io/projected/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-kube-api-access-9mtj4\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.426726 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426504 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-installation-pull-secrets\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426726 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.426533 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls podName:aca9032d-5d74-4374-a10f-694d6ced4025 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:23.926516449 +0000 UTC m=+35.636965673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls") pod "image-registry-7c6d6cc778-kcxhm" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025") : secret "image-registry-tls" not found Jun 02 05:26:23.426726 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426538 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-config-volume\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.426726 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426686 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-registry-certificates\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.426921 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.426802 2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 05:26:23.426921 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.426810 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 05:26:23.426921 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.426904 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert podName:7c31483d-0694-44d8-9125-f41ddf67e3d9 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:23.926884171 +0000 UTC m=+35.637333416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert") pod "ingress-canary-rw625" (UID: "7c31483d-0694-44d8-9125-f41ddf67e3d9") : secret "canary-serving-cert" not found Jun 02 05:26:23.427090 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.426927 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls podName:b1df4c97-bd0a-44e3-bc70-1c71520b0eec nodeName:}" failed. No retries permitted until 2026-06-02 05:26:23.926915796 +0000 UTC m=+35.637365022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls") pod "dns-default-kc7f9" (UID: "b1df4c97-bd0a-44e3-bc70-1c71520b0eec") : secret "dns-default-metrics-tls" not found Jun 02 05:26:23.427090 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.426991 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-tmp-dir\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.427385 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.427364 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-trusted-ca\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.431279 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.431253 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-image-registry-private-configuration\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.431425 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.431255 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-installation-pull-secrets\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.438824 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.438800 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-bound-sa-token\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.439178 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.439142 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slqjb\" (UniqueName: \"kubernetes.io/projected/7c31483d-0694-44d8-9125-f41ddf67e3d9-kube-api-access-slqjb\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:23.439425 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.439404 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mtj4\" (UniqueName: \"kubernetes.io/projected/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-kube-api-access-9mtj4\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.439678 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.439657 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqjrx\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-kube-api-access-kqjrx\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.527135 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.527094 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5fm5b\" (UniqueName: \"kubernetes.io/projected/16712eb9-39a2-48fe-aa0c-787be9e0577f-kube-api-access-5fm5b\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.527360 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.527153 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/16712eb9-39a2-48fe-aa0c-787be9e0577f-signing-cabundle\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.527360 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.527266 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/16712eb9-39a2-48fe-aa0c-787be9e0577f-signing-key\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.527888 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.527869 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/16712eb9-39a2-48fe-aa0c-787be9e0577f-signing-cabundle\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.530130 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.530107 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/16712eb9-39a2-48fe-aa0c-787be9e0577f-signing-key\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.537926 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.537906 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fm5b\" (UniqueName: \"kubernetes.io/projected/16712eb9-39a2-48fe-aa0c-787be9e0577f-kube-api-access-5fm5b\") pod \"service-ca-685df94d57-j8zz4\" (UID: \"16712eb9-39a2-48fe-aa0c-787be9e0577f\") " pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.658936 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.658901 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xgbc5"] Jun 02 05:26:23.663846 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.663819 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-j8zz4" Jun 02 05:26:23.679227 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.679149 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xgbc5"] Jun 02 05:26:23.679359 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.679247 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.681785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.681756 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 05:26:23.682109 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.682094 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 05:26:23.682337 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.682321 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 05:26:23.682510 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.682493 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 05:26:23.682690 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.682675 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sktqn\"" Jun 02 05:26:23.782057 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.782025 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:23.782057 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.782057 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:23.782331 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.782057 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:23.784845 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.784824 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 05:26:23.784984 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.784915 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 05:26:23.785092 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.785077 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 05:26:23.785427 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.785236 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qgj58\"" Jun 02 05:26:23.785427 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.785273 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8v96d\"" Jun 02 05:26:23.785427 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.785394 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 05:26:23.828568 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.828538 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7780cc-3632-457b-9023-8e60bf763765-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.828713 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.828603 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f7780cc-3632-457b-9023-8e60bf763765-data-volume\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.828713 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.828631 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjchd\" (UniqueName: \"kubernetes.io/projected/7f7780cc-3632-457b-9023-8e60bf763765-kube-api-access-rjchd\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.828713 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.828651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f7780cc-3632-457b-9023-8e60bf763765-crio-socket\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.828853 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.828737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.929642 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.929548 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:23.929642 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.929599 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f7780cc-3632-457b-9023-8e60bf763765-data-volume\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.929642 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.929628 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rjchd\" (UniqueName: \"kubernetes.io/projected/7f7780cc-3632-457b-9023-8e60bf763765-kube-api-access-rjchd\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.929661 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f7780cc-3632-457b-9023-8e60bf763765-crio-socket\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.929700 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.929717 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.929776 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls podName:b1df4c97-bd0a-44e3-bc70-1c71520b0eec nodeName:}" failed. No retries permitted until 2026-06-02 05:26:24.929756224 +0000 UTC m=+36.640205449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls") pod "dns-default-kc7f9" (UID: "b1df4c97-bd0a-44e3-bc70-1c71520b0eec") : secret "dns-default-metrics-tls" not found Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.929804 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.929824 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.929850 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7780cc-3632-457b-9023-8e60bf763765-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.929884 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls podName:7f7780cc-3632-457b-9023-8e60bf763765 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:24.429867897 +0000 UTC m=+36.140317121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgbc5" (UID: "7f7780cc-3632-457b-9023-8e60bf763765") : secret "insights-runtime-extractor-tls" not found Jun 02 05:26:23.929999 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.929904 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:23.930466 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.930009 2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 05:26:23.930466 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.930042 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert podName:7c31483d-0694-44d8-9125-f41ddf67e3d9 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:24.930031081 +0000 UTC m=+36.640480310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert") pod "ingress-canary-rw625" (UID: "7c31483d-0694-44d8-9125-f41ddf67e3d9") : secret "canary-serving-cert" not found Jun 02 05:26:23.930466 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.930192 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f7780cc-3632-457b-9023-8e60bf763765-crio-socket\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.930466 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.930433 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f7780cc-3632-457b-9023-8e60bf763765-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.930662 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.930495 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f7780cc-3632-457b-9023-8e60bf763765-data-volume\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:23.930662 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.930522 2561 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 05:26:23.930662 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.930536 2561 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6d6cc778-kcxhm: secret "image-registry-tls" not found Jun 02 05:26:23.930662 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:23.930575 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls podName:aca9032d-5d74-4374-a10f-694d6ced4025 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:24.93056044 +0000 UTC m=+36.641009681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls") pod "image-registry-7c6d6cc778-kcxhm" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025") : secret "image-registry-tls" not found Jun 02 05:26:23.939371 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:23.939340 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjchd\" (UniqueName: \"kubernetes.io/projected/7f7780cc-3632-457b-9023-8e60bf763765-kube-api-access-rjchd\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:24.434352 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:24.434310 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:24.434877 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.434489 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 05:26:24.434877 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.434580 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls podName:7f7780cc-3632-457b-9023-8e60bf763765 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:25.434559896 +0000 UTC m=+37.145009141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgbc5" (UID: "7f7780cc-3632-457b-9023-8e60bf763765") : secret "insights-runtime-extractor-tls" not found Jun 02 05:26:24.938195 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:24.938150 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:24.938371 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:24.938218 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:24.938371 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.938316 2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 05:26:24.938371 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:24.938331 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:24.938371 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.938368 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 05:26:24.938560 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.938384 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert podName:7c31483d-0694-44d8-9125-f41ddf67e3d9 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:26.938364407 +0000 UTC m=+38.648813636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert") pod "ingress-canary-rw625" (UID: "7c31483d-0694-44d8-9125-f41ddf67e3d9") : secret "canary-serving-cert" not found Jun 02 05:26:24.938560 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.938405 2561 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 05:26:24.938560 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.938417 2561 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6d6cc778-kcxhm: secret "image-registry-tls" not found Jun 02 05:26:24.938560 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.938418 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls podName:b1df4c97-bd0a-44e3-bc70-1c71520b0eec nodeName:}" failed. No retries permitted until 2026-06-02 05:26:26.938404761 +0000 UTC m=+38.648853985 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls") pod "dns-default-kc7f9" (UID: "b1df4c97-bd0a-44e3-bc70-1c71520b0eec") : secret "dns-default-metrics-tls" not found Jun 02 05:26:24.938560 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:24.938465 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls podName:aca9032d-5d74-4374-a10f-694d6ced4025 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:26.938451205 +0000 UTC m=+38.648900429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls") pod "image-registry-7c6d6cc778-kcxhm" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025") : secret "image-registry-tls" not found Jun 02 05:26:25.211824 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:25.211749 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-j8zz4"] Jun 02 05:26:25.247613 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:25.247579 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16712eb9_39a2_48fe_aa0c_787be9e0577f.slice/crio-77d237cdc310f80ffe82770d4ecb2c47f6de597acccea1dd9712d254f9278fdf WatchSource:0}: Error finding container 77d237cdc310f80ffe82770d4ecb2c47f6de597acccea1dd9712d254f9278fdf: Status 404 returned error can't find the container with id 77d237cdc310f80ffe82770d4ecb2c47f6de597acccea1dd9712d254f9278fdf Jun 02 05:26:25.442353 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:25.442306 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:25.442822 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:25.442478 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 05:26:25.442822 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:25.442569 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls podName:7f7780cc-3632-457b-9023-8e60bf763765 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:27.442545566 +0000 UTC m=+39.152994806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgbc5" (UID: "7f7780cc-3632-457b-9023-8e60bf763765") : secret "insights-runtime-extractor-tls" not found Jun 02 05:26:25.924747 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:25.924709 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-j8zz4" event={"ID":"16712eb9-39a2-48fe-aa0c-787be9e0577f","Type":"ContainerStarted","Data":"77d237cdc310f80ffe82770d4ecb2c47f6de597acccea1dd9712d254f9278fdf"} Jun 02 05:26:25.927562 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:25.927529 2561 generic.go:358] "Generic (PLEG): container finished" podID="420349fd-ebb4-48ed-8c9f-3a60a886d66c" containerID="c465ad98060f3c05ef1f8b9bff05e4a17c1482a244eba32982102e057b41f6f7" exitCode=0 Jun 02 05:26:25.927682 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:25.927568 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" event={"ID":"420349fd-ebb4-48ed-8c9f-3a60a886d66c","Type":"ContainerDied","Data":"c465ad98060f3c05ef1f8b9bff05e4a17c1482a244eba32982102e057b41f6f7"} Jun 02 05:26:26.932822 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:26.932626 2561 generic.go:358] "Generic (PLEG): container finished" podID="420349fd-ebb4-48ed-8c9f-3a60a886d66c" containerID="f5a7052e7ea3860590a0d75e2eb174ef8ae11cf4383a2c6e714b31533f5bcd6f" exitCode=0 Jun 02 05:26:26.933236 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:26.932713 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" event={"ID":"420349fd-ebb4-48ed-8c9f-3a60a886d66c","Type":"ContainerDied","Data":"f5a7052e7ea3860590a0d75e2eb174ef8ae11cf4383a2c6e714b31533f5bcd6f"} Jun 02 05:26:26.954976 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:26.954937 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:26.955118 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:26.955006 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:26.955118 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:26.955044 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:26.955118 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:26.955074 2561 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 05:26:26.955118 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:26.955092 2561 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6d6cc778-kcxhm: secret "image-registry-tls" not found Jun 02 05:26:26.955288 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:26.955145 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls podName:aca9032d-5d74-4374-a10f-694d6ced4025 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:30.955127003 +0000 UTC m=+42.665576229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls") pod "image-registry-7c6d6cc778-kcxhm" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025") : secret "image-registry-tls" not found Jun 02 05:26:26.955288 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:26.955163 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 05:26:26.955288 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:26.955168 2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 05:26:26.955288 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:26.955195 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls podName:b1df4c97-bd0a-44e3-bc70-1c71520b0eec nodeName:}" failed. No retries permitted until 2026-06-02 05:26:30.95518517 +0000 UTC m=+42.665634409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls") pod "dns-default-kc7f9" (UID: "b1df4c97-bd0a-44e3-bc70-1c71520b0eec") : secret "dns-default-metrics-tls" not found Jun 02 05:26:26.955288 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:26.955228 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert podName:7c31483d-0694-44d8-9125-f41ddf67e3d9 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:30.955210366 +0000 UTC m=+42.665659605 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert") pod "ingress-canary-rw625" (UID: "7c31483d-0694-44d8-9125-f41ddf67e3d9") : secret "canary-serving-cert" not found Jun 02 05:26:27.460503 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:27.460468 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:27.460620 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:27.460603 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 05:26:27.460678 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:27.460668 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls podName:7f7780cc-3632-457b-9023-8e60bf763765 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:31.460648772 +0000 UTC m=+43.171097996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgbc5" (UID: "7f7780cc-3632-457b-9023-8e60bf763765") : secret "insights-runtime-extractor-tls" not found Jun 02 05:26:27.936308 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:27.936272 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-j8zz4" event={"ID":"16712eb9-39a2-48fe-aa0c-787be9e0577f","Type":"ContainerStarted","Data":"85ddb92fac418395754b14b0c7dbae10eb89e06d3df5d55dc4ade3d67c00d661"} Jun 02 05:26:27.939181 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:27.939157 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" event={"ID":"420349fd-ebb4-48ed-8c9f-3a60a886d66c","Type":"ContainerStarted","Data":"e9287c85f74379aea907ac50ba744bbc19d17bd62cf2eea892033f9702ac55f0"} Jun 02 05:26:27.952451 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:27.951260 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-j8zz4" podStartSLOduration=2.841061715 podStartE2EDuration="4.951244867s" podCreationTimestamp="2026-06-02 05:26:23 +0000 UTC" firstStartedPulling="2026-06-02 05:26:25.255994945 +0000 UTC m=+36.966444169" lastFinishedPulling="2026-06-02 05:26:27.366178098 +0000 UTC m=+39.076627321" observedRunningTime="2026-06-02 05:26:27.9510846 +0000 UTC m=+39.661533847" watchObservedRunningTime="2026-06-02 05:26:27.951244867 +0000 UTC m=+39.661694114" Jun 02 05:26:27.972676 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:27.972639 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vq6j4" podStartSLOduration=6.089728354 podStartE2EDuration="39.972629311s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:25:51.394875508 +0000 UTC m=+3.105324737" lastFinishedPulling="2026-06-02 05:26:25.277776467 +0000 UTC m=+36.988225694" observedRunningTime="2026-06-02 05:26:27.97167879 +0000 UTC m=+39.682128034" watchObservedRunningTime="2026-06-02 05:26:27.972629311 +0000 UTC m=+39.683078556" Jun 02 05:26:30.988282 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:30.988243 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:30.988282 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:30.988288 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:30.988806 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:30.988311 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:30.988806 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:30.988394 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 05:26:30.988806 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:30.988396 2561 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 05:26:30.988806 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:30.988423 2561 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c6d6cc778-kcxhm: secret "image-registry-tls" not found Jun 02 05:26:30.988806 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:30.988442 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls podName:b1df4c97-bd0a-44e3-bc70-1c71520b0eec nodeName:}" failed. No retries permitted until 2026-06-02 05:26:38.988427638 +0000 UTC m=+50.698876862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls") pod "dns-default-kc7f9" (UID: "b1df4c97-bd0a-44e3-bc70-1c71520b0eec") : secret "dns-default-metrics-tls" not found Jun 02 05:26:30.988806 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:30.988463 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls podName:aca9032d-5d74-4374-a10f-694d6ced4025 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:38.988451291 +0000 UTC m=+50.698900516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls") pod "image-registry-7c6d6cc778-kcxhm" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025") : secret "image-registry-tls" not found Jun 02 05:26:30.988806 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:30.988400 2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 05:26:30.988806 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:30.988491 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert podName:7c31483d-0694-44d8-9125-f41ddf67e3d9 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:38.988481371 +0000 UTC m=+50.698930643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert") pod "ingress-canary-rw625" (UID: "7c31483d-0694-44d8-9125-f41ddf67e3d9") : secret "canary-serving-cert" not found Jun 02 05:26:31.491540 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:31.491507 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:31.491714 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:31.491674 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 05:26:31.491788 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:26:31.491754 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls podName:7f7780cc-3632-457b-9023-8e60bf763765 nodeName:}" failed. No retries permitted until 2026-06-02 05:26:39.491733148 +0000 UTC m=+51.202182394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgbc5" (UID: "7f7780cc-3632-457b-9023-8e60bf763765") : secret "insights-runtime-extractor-tls" not found Jun 02 05:26:33.712758 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:33.712724 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:33.716211 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:33.716187 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47e6d278-6f80-4c82-857d-564ed5515845-original-pull-secret\") pod \"global-pull-secret-syncer-8qmnz\" (UID: \"47e6d278-6f80-4c82-857d-564ed5515845\") " pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:34.001997 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:34.001902 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qmnz" Jun 02 05:26:34.142423 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:34.142392 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8qmnz"] Jun 02 05:26:34.146064 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:34.146027 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e6d278_6f80_4c82_857d_564ed5515845.slice/crio-dbd66bfeb6cd5a194533ad26fecb6329756b9c29ea134fa3ea35e93b0552f754 WatchSource:0}: Error finding container dbd66bfeb6cd5a194533ad26fecb6329756b9c29ea134fa3ea35e93b0552f754: Status 404 returned error can't find the container with id dbd66bfeb6cd5a194533ad26fecb6329756b9c29ea134fa3ea35e93b0552f754 Jun 02 05:26:34.956510 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:34.956458 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8qmnz" event={"ID":"47e6d278-6f80-4c82-857d-564ed5515845","Type":"ContainerStarted","Data":"dbd66bfeb6cd5a194533ad26fecb6329756b9c29ea134fa3ea35e93b0552f754"} Jun 02 05:26:38.966394 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:38.966352 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8qmnz" event={"ID":"47e6d278-6f80-4c82-857d-564ed5515845","Type":"ContainerStarted","Data":"4f636f7f30e35804b0225add55552d75868265bfb457c1598fd215b0269ea440"} Jun 02 05:26:38.986679 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:38.986636 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8qmnz" podStartSLOduration=33.712381514 podStartE2EDuration="37.98662231s" podCreationTimestamp="2026-06-02 05:26:01 +0000 UTC" firstStartedPulling="2026-06-02 05:26:34.147866066 +0000 UTC m=+45.858315290" lastFinishedPulling="2026-06-02 05:26:38.422106858 +0000 UTC m=+50.132556086" observedRunningTime="2026-06-02 05:26:38.985866847 +0000 UTC m=+50.696316094" watchObservedRunningTime="2026-06-02 05:26:38.98662231 +0000 UTC m=+50.697071557" Jun 02 05:26:39.058657 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.058620 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:39.058657 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.058661 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:39.058886 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.058785 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:39.061267 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.061238 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"image-registry-7c6d6cc778-kcxhm\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:39.061385 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.061364 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1df4c97-bd0a-44e3-bc70-1c71520b0eec-metrics-tls\") pod \"dns-default-kc7f9\" (UID: \"b1df4c97-bd0a-44e3-bc70-1c71520b0eec\") " pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:39.061385 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.061372 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c31483d-0694-44d8-9125-f41ddf67e3d9-cert\") pod \"ingress-canary-rw625\" (UID: \"7c31483d-0694-44d8-9125-f41ddf67e3d9\") " pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:39.128021 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.127983 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:39.147944 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.147917 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:39.155140 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.155116 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rw625" Jun 02 05:26:39.275979 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.275928 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c6d6cc778-kcxhm"] Jun 02 05:26:39.278812 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:39.278783 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca9032d_5d74_4374_a10f_694d6ced4025.slice/crio-16affdaf5925a76692d2b1587c8296890af6d295920205dd4781dbd12f6604d7 WatchSource:0}: Error finding container 16affdaf5925a76692d2b1587c8296890af6d295920205dd4781dbd12f6604d7: Status 404 returned error can't find the container with id 16affdaf5925a76692d2b1587c8296890af6d295920205dd4781dbd12f6604d7 Jun 02 05:26:39.295826 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.295799 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kc7f9"] Jun 02 05:26:39.299596 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:39.299565 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1df4c97_bd0a_44e3_bc70_1c71520b0eec.slice/crio-b29feb96a050d0968ae26c92d7ab0332743642af849beb97ba2ae228b58137e5 WatchSource:0}: Error finding container b29feb96a050d0968ae26c92d7ab0332743642af849beb97ba2ae228b58137e5: Status 404 returned error can't find the container with id b29feb96a050d0968ae26c92d7ab0332743642af849beb97ba2ae228b58137e5 Jun 02 05:26:39.307158 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.307135 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rw625"] Jun 02 05:26:39.311013 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:39.310990 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c31483d_0694_44d8_9125_f41ddf67e3d9.slice/crio-6c18d61f652ddc693b4fa733d2b1ec88e825bc382266244b6236488a9d7c0507 WatchSource:0}: Error finding container 6c18d61f652ddc693b4fa733d2b1ec88e825bc382266244b6236488a9d7c0507: Status 404 returned error can't find the container with id 6c18d61f652ddc693b4fa733d2b1ec88e825bc382266244b6236488a9d7c0507 Jun 02 05:26:39.563461 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.563420 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:39.565749 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.565730 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f7780cc-3632-457b-9023-8e60bf763765-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgbc5\" (UID: \"7f7780cc-3632-457b-9023-8e60bf763765\") " pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:39.590719 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.590694 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xgbc5" Jun 02 05:26:39.735230 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.735193 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xgbc5"] Jun 02 05:26:39.971765 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.971732 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rw625" event={"ID":"7c31483d-0694-44d8-9125-f41ddf67e3d9","Type":"ContainerStarted","Data":"6c18d61f652ddc693b4fa733d2b1ec88e825bc382266244b6236488a9d7c0507"} Jun 02 05:26:39.973785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.973756 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc7f9" event={"ID":"b1df4c97-bd0a-44e3-bc70-1c71520b0eec","Type":"ContainerStarted","Data":"b29feb96a050d0968ae26c92d7ab0332743642af849beb97ba2ae228b58137e5"} Jun 02 05:26:39.975390 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.975363 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" event={"ID":"aca9032d-5d74-4374-a10f-694d6ced4025","Type":"ContainerStarted","Data":"637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6"} Jun 02 05:26:39.975482 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.975419 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" event={"ID":"aca9032d-5d74-4374-a10f-694d6ced4025","Type":"ContainerStarted","Data":"16affdaf5925a76692d2b1587c8296890af6d295920205dd4781dbd12f6604d7"} Jun 02 05:26:39.975553 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.975482 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:39.977752 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.977732 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgbc5" event={"ID":"7f7780cc-3632-457b-9023-8e60bf763765","Type":"ContainerStarted","Data":"6a42ee406de9bce61efab5b983ca47d332cbbfc9afcf3ef389e48b00b6ec5145"} Jun 02 05:26:39.977830 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.977757 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgbc5" event={"ID":"7f7780cc-3632-457b-9023-8e60bf763765","Type":"ContainerStarted","Data":"37283f1bdd8443096f1883f537679572b24d26f092d717561d725b37b5b6e435"} Jun 02 05:26:39.996605 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:39.996562 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" podStartSLOduration=63.996547329 podStartE2EDuration="1m3.996547329s" podCreationTimestamp="2026-06-02 05:25:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:26:39.995221957 +0000 UTC m=+51.705671205" watchObservedRunningTime="2026-06-02 05:26:39.996547329 +0000 UTC m=+51.706996592" Jun 02 05:26:41.988033 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:41.987998 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc7f9" event={"ID":"b1df4c97-bd0a-44e3-bc70-1c71520b0eec","Type":"ContainerStarted","Data":"06ebd10310f919787a2f474199e72496eb6e49ad3e11da2f54d34b9e513ec7d8"} Jun 02 05:26:41.989933 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:41.989897 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgbc5" event={"ID":"7f7780cc-3632-457b-9023-8e60bf763765","Type":"ContainerStarted","Data":"0c927a5f5ef630e914e64cc85c74f27b7771a369e9521a2cbcc4d57c51276712"} Jun 02 05:26:41.990952 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:41.990932 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rw625" event={"ID":"7c31483d-0694-44d8-9125-f41ddf67e3d9","Type":"ContainerStarted","Data":"86cd5e52bce1878820436fa6b5aa60cde48e42501a10d6dd2f40fab220c9274d"} Jun 02 05:26:42.008286 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:42.008062 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rw625" podStartSLOduration=16.489215665 podStartE2EDuration="19.008050406s" podCreationTimestamp="2026-06-02 05:26:23 +0000 UTC" firstStartedPulling="2026-06-02 05:26:39.312935937 +0000 UTC m=+51.023385162" lastFinishedPulling="2026-06-02 05:26:41.831770679 +0000 UTC m=+53.542219903" observedRunningTime="2026-06-02 05:26:42.007287908 +0000 UTC m=+53.717737153" watchObservedRunningTime="2026-06-02 05:26:42.008050406 +0000 UTC m=+53.718499651" Jun 02 05:26:42.995940 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:42.995899 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc7f9" event={"ID":"b1df4c97-bd0a-44e3-bc70-1c71520b0eec","Type":"ContainerStarted","Data":"0fd53a7689566f801c594401d04d7f48c4b5574062e5a068790852ef9763b5ed"} Jun 02 05:26:43.016552 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:43.016227 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-kc7f9" podStartSLOduration=17.488786649 podStartE2EDuration="20.016209058s" podCreationTimestamp="2026-06-02 05:26:23 +0000 UTC" firstStartedPulling="2026-06-02 05:26:39.301369321 +0000 UTC m=+51.011818545" lastFinishedPulling="2026-06-02 05:26:41.828791728 +0000 UTC m=+53.539240954" observedRunningTime="2026-06-02 05:26:43.01556658 +0000 UTC m=+54.726015826" watchObservedRunningTime="2026-06-02 05:26:43.016209058 +0000 UTC m=+54.726658325" Jun 02 05:26:44.001014 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:44.000958 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgbc5" event={"ID":"7f7780cc-3632-457b-9023-8e60bf763765","Type":"ContainerStarted","Data":"6de0b033daf01bc55ace64a641ad001666a6e60ff59339ef10f879b62ec6d0a6"} Jun 02 05:26:44.001414 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:44.001300 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:44.019477 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:44.019424 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xgbc5" podStartSLOduration=17.465352922 podStartE2EDuration="21.01940823s" podCreationTimestamp="2026-06-02 05:26:23 +0000 UTC" firstStartedPulling="2026-06-02 05:26:39.836550944 +0000 UTC m=+51.547000169" lastFinishedPulling="2026-06-02 05:26:43.39060625 +0000 UTC m=+55.101055477" observedRunningTime="2026-06-02 05:26:44.019205305 +0000 UTC m=+55.729654550" watchObservedRunningTime="2026-06-02 05:26:44.01940823 +0000 UTC m=+55.729857479" Jun 02 05:26:49.484676 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.484642 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d76657986-fp4z5"] Jun 02 05:26:49.487882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.487864 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-cqwx6"] Jun 02 05:26:49.488053 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.488027 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.490177 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.490151 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-cqwx6" Jun 02 05:26:49.493609 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.493591 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 05:26:49.493609 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.493601 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 05:26:49.493716 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.493631 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 05:26:49.494513 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.494326 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 05:26:49.494513 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.494347 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 05:26:49.494513 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.494414 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vmd8c\"" Jun 02 05:26:49.495138 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.495120 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 05:26:49.495809 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.495792 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 05:26:49.496414 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.496399 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8j8r8\"" Jun 02 05:26:49.499886 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.499850 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c6d6cc778-kcxhm"] Jun 02 05:26:49.504802 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.504738 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-cqwx6"] Jun 02 05:26:49.506296 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.506273 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d76657986-fp4z5"] Jun 02 05:26:49.536096 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.536073 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-console-config\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.536211 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.536117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-oauth-serving-cert\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.536211 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.536171 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4kmp\" (UniqueName: \"kubernetes.io/projected/2b70cc86-eebd-4e20-961b-743fead30181-kube-api-access-p4kmp\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.536211 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.536202 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99hjx\" (UniqueName: \"kubernetes.io/projected/8abf55f0-6c9e-4be7-a98b-1f7854ecab74-kube-api-access-99hjx\") pod \"downloads-c65b8dc4b-cqwx6\" (UID: \"8abf55f0-6c9e-4be7-a98b-1f7854ecab74\") " pod="openshift-console/downloads-c65b8dc4b-cqwx6" Jun 02 05:26:49.536337 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.536243 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-oauth-config\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.536337 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.536304 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-serving-cert\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.536407 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.536345 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-service-ca\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.569173 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.569144 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6658fc4b86-t7pgh"] Jun 02 05:26:49.571514 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.571500 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.590445 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.590428 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6658fc4b86-t7pgh"] Jun 02 05:26:49.637179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637153 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-ca-trust-extracted\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.637179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637184 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-registry-tls\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.637389 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637207 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-console-config\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.637389 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637225 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm4wh\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-kube-api-access-cm4wh\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.637389 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637275 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-oauth-serving-cert\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.637389 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637297 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p4kmp\" (UniqueName: \"kubernetes.io/projected/2b70cc86-eebd-4e20-961b-743fead30181-kube-api-access-p4kmp\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.637389 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637317 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-99hjx\" (UniqueName: \"kubernetes.io/projected/8abf55f0-6c9e-4be7-a98b-1f7854ecab74-kube-api-access-99hjx\") pod \"downloads-c65b8dc4b-cqwx6\" (UID: \"8abf55f0-6c9e-4be7-a98b-1f7854ecab74\") " pod="openshift-console/downloads-c65b8dc4b-cqwx6" Jun 02 05:26:49.637389 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637341 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-registry-certificates\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.637389 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637366 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-bound-sa-token\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.637698 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-trusted-ca\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.637698 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637419 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-oauth-config\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.637698 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637510 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-serving-cert\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.637698 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637579 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-image-registry-private-configuration\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.637698 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637610 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-service-ca\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.637698 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637653 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-installation-pull-secrets\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.638033 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637936 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-oauth-serving-cert\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.638033 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.637946 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-console-config\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.638194 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.638177 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-service-ca\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.639888 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.639867 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-oauth-config\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.640124 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.640098 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-serving-cert\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.652689 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.652664 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99hjx\" (UniqueName: \"kubernetes.io/projected/8abf55f0-6c9e-4be7-a98b-1f7854ecab74-kube-api-access-99hjx\") pod \"downloads-c65b8dc4b-cqwx6\" (UID: \"8abf55f0-6c9e-4be7-a98b-1f7854ecab74\") " pod="openshift-console/downloads-c65b8dc4b-cqwx6" Jun 02 05:26:49.653071 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.653054 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4kmp\" (UniqueName: \"kubernetes.io/projected/2b70cc86-eebd-4e20-961b-743fead30181-kube-api-access-p4kmp\") pod \"console-5d76657986-fp4z5\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.738649 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.738577 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-registry-tls\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.738649 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.738612 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4wh\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-kube-api-access-cm4wh\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.738835 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.738650 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-registry-certificates\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.738835 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.738666 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-bound-sa-token\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.738835 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.738688 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-trusted-ca\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.738835 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.738740 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-image-registry-private-configuration\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.738835 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.738767 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-installation-pull-secrets\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.738835 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.738794 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-ca-trust-extracted\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.739323 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.739296 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-ca-trust-extracted\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.739904 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.739861 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-registry-certificates\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.739904 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.739888 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-trusted-ca\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.741145 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.741117 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-registry-tls\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.741351 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.741335 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-installation-pull-secrets\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.741426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.741407 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-image-registry-private-configuration\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.749358 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.749341 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-bound-sa-token\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.750418 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.750394 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm4wh\" (UniqueName: \"kubernetes.io/projected/b05a3131-07de-4724-a7bd-e6ee4c6b6dbd-kube-api-access-cm4wh\") pod \"image-registry-6658fc4b86-t7pgh\" (UID: \"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd\") " pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.797661 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.797637 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:49.806376 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.806357 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-cqwx6" Jun 02 05:26:49.879528 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.879503 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:49.926042 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.926010 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tz8dl" Jun 02 05:26:49.948118 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.948088 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d76657986-fp4z5"] Jun 02 05:26:49.952227 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:49.952195 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b70cc86_eebd_4e20_961b_743fead30181.slice/crio-823f08c1e1d79b3ce43771523bef0b5bbe8565e44f3e5651660d08207ba1f7ea WatchSource:0}: Error finding container 823f08c1e1d79b3ce43771523bef0b5bbe8565e44f3e5651660d08207ba1f7ea: Status 404 returned error can't find the container with id 823f08c1e1d79b3ce43771523bef0b5bbe8565e44f3e5651660d08207ba1f7ea Jun 02 05:26:49.983166 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:49.983133 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-cqwx6"] Jun 02 05:26:49.989016 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:49.988946 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8abf55f0_6c9e_4be7_a98b_1f7854ecab74.slice/crio-81771c18dd229035c17e084d8e3367d9e731da9086a718481d774c83cc5eba2d WatchSource:0}: Error finding container 81771c18dd229035c17e084d8e3367d9e731da9086a718481d774c83cc5eba2d: Status 404 returned error can't find the container with id 81771c18dd229035c17e084d8e3367d9e731da9086a718481d774c83cc5eba2d Jun 02 05:26:50.018905 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:50.018871 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-cqwx6" event={"ID":"8abf55f0-6c9e-4be7-a98b-1f7854ecab74","Type":"ContainerStarted","Data":"81771c18dd229035c17e084d8e3367d9e731da9086a718481d774c83cc5eba2d"} Jun 02 05:26:50.019900 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:50.019875 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d76657986-fp4z5" event={"ID":"2b70cc86-eebd-4e20-961b-743fead30181","Type":"ContainerStarted","Data":"823f08c1e1d79b3ce43771523bef0b5bbe8565e44f3e5651660d08207ba1f7ea"} Jun 02 05:26:50.035521 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:50.035493 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6658fc4b86-t7pgh"] Jun 02 05:26:50.039104 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:50.039081 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05a3131_07de_4724_a7bd_e6ee4c6b6dbd.slice/crio-285ae278e89a9dc386176915917304f4e9330f322a962e5cd81ee5c24af0726f WatchSource:0}: Error finding container 285ae278e89a9dc386176915917304f4e9330f322a962e5cd81ee5c24af0726f: Status 404 returned error can't find the container with id 285ae278e89a9dc386176915917304f4e9330f322a962e5cd81ee5c24af0726f Jun 02 05:26:51.006120 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:51.006086 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kc7f9" Jun 02 05:26:51.025979 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:51.025931 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" event={"ID":"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd","Type":"ContainerStarted","Data":"db7090ca5140c496753c13d591955e37bcdf0672c029c7b22e28da01b7f117f2"} Jun 02 05:26:51.026176 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:51.025998 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" event={"ID":"b05a3131-07de-4724-a7bd-e6ee4c6b6dbd","Type":"ContainerStarted","Data":"285ae278e89a9dc386176915917304f4e9330f322a962e5cd81ee5c24af0726f"} Jun 02 05:26:51.026362 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:51.026336 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:26:54.482065 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.482028 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:54.484641 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.484624 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 05:26:54.495240 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.495221 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99b8d9f-7723-40bc-be05-b17d605888b4-metrics-certs\") pod \"network-metrics-daemon-4fgvs\" (UID: \"a99b8d9f-7723-40bc-be05-b17d605888b4\") " pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:54.683227 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.683193 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:54.685788 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.685759 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 05:26:54.696499 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.696332 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 05:26:54.696499 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.696377 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8v96d\"" Jun 02 05:26:54.704450 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.704423 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fgvs" Jun 02 05:26:54.707336 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.707315 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p24nd\" (UniqueName: \"kubernetes.io/projected/ddfa33ef-1107-42ae-b2aa-b281becf554b-kube-api-access-p24nd\") pod \"network-check-target-9vkts\" (UID: \"ddfa33ef-1107-42ae-b2aa-b281becf554b\") " pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:54.836220 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.836168 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" podStartSLOduration=5.8361515189999995 podStartE2EDuration="5.836151519s" podCreationTimestamp="2026-06-02 05:26:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:26:51.048910863 +0000 UTC m=+62.759360110" watchObservedRunningTime="2026-06-02 05:26:54.836151519 +0000 UTC m=+66.546600762" Jun 02 05:26:54.836690 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:54.836673 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4fgvs"] Jun 02 05:26:54.841225 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:54.841191 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99b8d9f_7723_40bc_be05_b17d605888b4.slice/crio-d21bf38018ccf23360eb7bd174a6077ddd930e0cbcda237b659066ff658621d6 WatchSource:0}: Error finding container d21bf38018ccf23360eb7bd174a6077ddd930e0cbcda237b659066ff658621d6: Status 404 returned error can't find the container with id d21bf38018ccf23360eb7bd174a6077ddd930e0cbcda237b659066ff658621d6 Jun 02 05:26:55.009324 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:55.009239 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qgj58\"" Jun 02 05:26:55.017487 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:55.017458 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:26:55.041287 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:55.041251 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d76657986-fp4z5" event={"ID":"2b70cc86-eebd-4e20-961b-743fead30181","Type":"ContainerStarted","Data":"57863195339ef9e8e84b7d46078afc1730ab21690293d9384a14c7154d57915c"} Jun 02 05:26:55.042577 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:55.042555 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4fgvs" event={"ID":"a99b8d9f-7723-40bc-be05-b17d605888b4","Type":"ContainerStarted","Data":"d21bf38018ccf23360eb7bd174a6077ddd930e0cbcda237b659066ff658621d6"} Jun 02 05:26:55.062905 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:55.062850 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5d76657986-fp4z5" podStartSLOduration=2.035185047 podStartE2EDuration="6.062834645s" podCreationTimestamp="2026-06-02 05:26:49 +0000 UTC" firstStartedPulling="2026-06-02 05:26:49.954492883 +0000 UTC m=+61.664942107" lastFinishedPulling="2026-06-02 05:26:53.982142478 +0000 UTC m=+65.692591705" observedRunningTime="2026-06-02 05:26:55.060366682 +0000 UTC m=+66.770815929" watchObservedRunningTime="2026-06-02 05:26:55.062834645 +0000 UTC m=+66.773283896" Jun 02 05:26:55.163049 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:55.163006 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9vkts"] Jun 02 05:26:55.166479 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:26:55.166446 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfa33ef_1107_42ae_b2aa_b281becf554b.slice/crio-d74dce235755a3e3ac857925fee5a30d00f202487a49aaed38e6589ad27bda32 WatchSource:0}: Error finding container d74dce235755a3e3ac857925fee5a30d00f202487a49aaed38e6589ad27bda32: Status 404 returned error can't find the container with id d74dce235755a3e3ac857925fee5a30d00f202487a49aaed38e6589ad27bda32 Jun 02 05:26:56.048032 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:56.047986 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9vkts" event={"ID":"ddfa33ef-1107-42ae-b2aa-b281becf554b","Type":"ContainerStarted","Data":"d74dce235755a3e3ac857925fee5a30d00f202487a49aaed38e6589ad27bda32"} Jun 02 05:26:57.053596 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:57.053559 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4fgvs" event={"ID":"a99b8d9f-7723-40bc-be05-b17d605888b4","Type":"ContainerStarted","Data":"5eb93d7cdcb5e6866476083051008984ca2468e64c879f4dc08d45325fc1db80"} Jun 02 05:26:57.053596 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:57.053600 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4fgvs" event={"ID":"a99b8d9f-7723-40bc-be05-b17d605888b4","Type":"ContainerStarted","Data":"a9e8b4246270d402f101df8c10094027a1f1de01394e7fa368a060f8aac861c9"} Jun 02 05:26:57.072448 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:57.072364 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4fgvs" podStartSLOduration=67.362999335 podStartE2EDuration="1m9.072350479s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:26:54.843025232 +0000 UTC m=+66.553474456" lastFinishedPulling="2026-06-02 05:26:56.552376371 +0000 UTC m=+68.262825600" observedRunningTime="2026-06-02 05:26:57.070838569 +0000 UTC m=+68.781287815" watchObservedRunningTime="2026-06-02 05:26:57.072350479 +0000 UTC m=+68.782799725" Jun 02 05:26:59.507899 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:59.507813 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:26:59.798352 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:59.798313 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:59.798532 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:59.798405 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:26:59.803900 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:26:59.803881 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:27:00.067305 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:00.067211 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9vkts" event={"ID":"ddfa33ef-1107-42ae-b2aa-b281becf554b","Type":"ContainerStarted","Data":"c7f8992331c3af9aa618dd0b9266430ec03ab05eb51750e1a579aacbd844b958"} Jun 02 05:27:00.067469 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:00.067433 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:27:00.072744 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:00.072720 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:27:00.083732 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:00.083686 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9vkts" podStartSLOduration=68.094334494 podStartE2EDuration="1m12.08367086s" podCreationTimestamp="2026-06-02 05:25:48 +0000 UTC" firstStartedPulling="2026-06-02 05:26:55.16860801 +0000 UTC m=+66.879057234" lastFinishedPulling="2026-06-02 05:26:59.15794437 +0000 UTC m=+70.868393600" observedRunningTime="2026-06-02 05:27:00.083592562 +0000 UTC m=+71.794041809" watchObservedRunningTime="2026-06-02 05:27:00.08367086 +0000 UTC m=+71.794120110" Jun 02 05:27:05.697730 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:05.697611 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d76657986-fp4z5"] Jun 02 05:27:10.106418 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:10.106378 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-cqwx6" event={"ID":"8abf55f0-6c9e-4be7-a98b-1f7854ecab74","Type":"ContainerStarted","Data":"e2c3c8f76d07058ffa4c5dea787feae9b91581438500a793b55d842455913b75"} Jun 02 05:27:10.106870 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:10.106637 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-cqwx6" Jun 02 05:27:10.119393 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:10.119369 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-cqwx6" Jun 02 05:27:10.136300 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:10.136243 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-cqwx6" podStartSLOduration=1.841209687 podStartE2EDuration="21.136228676s" podCreationTimestamp="2026-06-02 05:26:49 +0000 UTC" firstStartedPulling="2026-06-02 05:26:49.990733655 +0000 UTC m=+61.701182882" lastFinishedPulling="2026-06-02 05:27:09.285752645 +0000 UTC m=+80.996201871" observedRunningTime="2026-06-02 05:27:10.134643819 +0000 UTC m=+81.845093078" watchObservedRunningTime="2026-06-02 05:27:10.136228676 +0000 UTC m=+81.846677935" Jun 02 05:27:12.035240 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:12.035199 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6658fc4b86-t7pgh" Jun 02 05:27:14.524338 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.524274 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" podUID="aca9032d-5d74-4374-a10f-694d6ced4025" containerName="registry" containerID="cri-o://637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6" gracePeriod=30 Jun 02 05:27:14.818142 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.818115 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:27:14.959265 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.959223 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") pod \"aca9032d-5d74-4374-a10f-694d6ced4025\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " Jun 02 05:27:14.959449 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.959285 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-image-registry-private-configuration\") pod \"aca9032d-5d74-4374-a10f-694d6ced4025\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " Jun 02 05:27:14.959449 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.959341 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-trusted-ca\") pod \"aca9032d-5d74-4374-a10f-694d6ced4025\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " Jun 02 05:27:14.959449 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.959370 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-bound-sa-token\") pod \"aca9032d-5d74-4374-a10f-694d6ced4025\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " Jun 02 05:27:14.959449 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.959406 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqjrx\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-kube-api-access-kqjrx\") pod \"aca9032d-5d74-4374-a10f-694d6ced4025\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " Jun 02 05:27:14.959449 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.959441 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-installation-pull-secrets\") pod \"aca9032d-5d74-4374-a10f-694d6ced4025\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " Jun 02 05:27:14.959704 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.959470 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aca9032d-5d74-4374-a10f-694d6ced4025-ca-trust-extracted\") pod \"aca9032d-5d74-4374-a10f-694d6ced4025\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " Jun 02 05:27:14.959704 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.959500 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-registry-certificates\") pod \"aca9032d-5d74-4374-a10f-694d6ced4025\" (UID: \"aca9032d-5d74-4374-a10f-694d6ced4025\") " Jun 02 05:27:14.960042 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.960014 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "aca9032d-5d74-4374-a10f-694d6ced4025" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 05:27:14.960214 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.960042 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "aca9032d-5d74-4374-a10f-694d6ced4025" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 05:27:14.963316 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.963271 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "aca9032d-5d74-4374-a10f-694d6ced4025" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 05:27:14.963545 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.963513 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-kube-api-access-kqjrx" (OuterVolumeSpecName: "kube-api-access-kqjrx") pod "aca9032d-5d74-4374-a10f-694d6ced4025" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025"). InnerVolumeSpecName "kube-api-access-kqjrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:27:14.966334 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.966307 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "aca9032d-5d74-4374-a10f-694d6ced4025" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:27:14.966426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.966347 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "aca9032d-5d74-4374-a10f-694d6ced4025" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 05:27:14.966604 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.966580 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "aca9032d-5d74-4374-a10f-694d6ced4025" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:27:14.972112 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:14.972085 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aca9032d-5d74-4374-a10f-694d6ced4025-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "aca9032d-5d74-4374-a10f-694d6ced4025" (UID: "aca9032d-5d74-4374-a10f-694d6ced4025"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 05:27:15.060939 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.060857 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-image-registry-private-configuration\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:15.060939 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.060892 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-trusted-ca\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:15.060939 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.060905 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-bound-sa-token\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:15.060939 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.060914 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqjrx\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-kube-api-access-kqjrx\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:15.060939 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.060930 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aca9032d-5d74-4374-a10f-694d6ced4025-installation-pull-secrets\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:15.061321 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.060945 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aca9032d-5d74-4374-a10f-694d6ced4025-ca-trust-extracted\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:15.061321 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.060980 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aca9032d-5d74-4374-a10f-694d6ced4025-registry-certificates\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:15.061321 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.060995 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aca9032d-5d74-4374-a10f-694d6ced4025-registry-tls\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:15.126761 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.126724 2561 generic.go:358] "Generic (PLEG): container finished" podID="aca9032d-5d74-4374-a10f-694d6ced4025" containerID="637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6" exitCode=0 Jun 02 05:27:15.126929 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.126798 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" Jun 02 05:27:15.126929 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.126810 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" event={"ID":"aca9032d-5d74-4374-a10f-694d6ced4025","Type":"ContainerDied","Data":"637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6"} Jun 02 05:27:15.126929 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.126861 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6d6cc778-kcxhm" event={"ID":"aca9032d-5d74-4374-a10f-694d6ced4025","Type":"ContainerDied","Data":"16affdaf5925a76692d2b1587c8296890af6d295920205dd4781dbd12f6604d7"} Jun 02 05:27:15.126929 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.126883 2561 scope.go:117] "RemoveContainer" containerID="637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6" Jun 02 05:27:15.135994 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.135950 2561 scope.go:117] "RemoveContainer" containerID="637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6" Jun 02 05:27:15.136529 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:27:15.136301 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6\": container with ID starting with 637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6 not found: ID does not exist" containerID="637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6" Jun 02 05:27:15.136529 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.136335 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6"} err="failed to get container status \"637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6\": rpc error: code = NotFound desc = could not find container \"637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6\": container with ID starting with 637606e37b6a53bc022ff4f08cd904b5f83490a4818b201e27ac1ccc601808a6 not found: ID does not exist" Jun 02 05:27:15.150733 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.150711 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c6d6cc778-kcxhm"] Jun 02 05:27:15.153591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:15.153564 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c6d6cc778-kcxhm"] Jun 02 05:27:16.786518 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:16.786477 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aca9032d-5d74-4374-a10f-694d6ced4025" path="/var/lib/kubelet/pods/aca9032d-5d74-4374-a10f-694d6ced4025/volumes" Jun 02 05:27:20.014341 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.014303 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-7cfhl"] Jun 02 05:27:20.014792 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.014621 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aca9032d-5d74-4374-a10f-694d6ced4025" containerName="registry" Jun 02 05:27:20.014792 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.014635 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="aca9032d-5d74-4374-a10f-694d6ced4025" containerName="registry" Jun 02 05:27:20.014792 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.014698 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="aca9032d-5d74-4374-a10f-694d6ced4025" containerName="registry" Jun 02 05:27:20.066851 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.066813 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-7cfhl"] Jun 02 05:27:20.066851 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.066849 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4"] Jun 02 05:27:20.067113 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.067022 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.069477 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.069449 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 02 05:27:20.070466 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.070215 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 05:27:20.070466 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.070332 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 05:27:20.070686 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.070557 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rbhf8\"" Jun 02 05:27:20.070758 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.070745 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 05:27:20.071135 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.070953 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 02 05:27:20.083575 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.083553 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rnmxr"] Jun 02 05:27:20.083753 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.083735 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.086954 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.086827 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 02 05:27:20.087083 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.086990 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 02 05:27:20.087356 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.087171 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p2446\"" Jun 02 05:27:20.087356 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.087252 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 02 05:27:20.106524 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.106504 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4"] Jun 02 05:27:20.106653 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.106641 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.109258 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.109239 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 05:27:20.110067 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.109444 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 05:27:20.110170 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.109680 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jvnjx\"" Jun 02 05:27:20.110170 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.109711 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 05:27:20.204247 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-metrics-client-ca\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.204424 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204256 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-tls\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.204424 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204317 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.204424 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204351 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.204424 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204377 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-textfile\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.204424 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204403 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c841b114-f106-47ab-a637-850293ff966e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.204635 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204443 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-wtmp\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.204635 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6vk5\" (UniqueName: \"kubernetes.io/projected/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-api-access-n6vk5\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.204635 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1021c7e-8fa9-4dac-b2be-17baa7deb980-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.204635 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204545 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c841b114-f106-47ab-a637-850293ff966e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.204635 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204577 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c841b114-f106-47ab-a637-850293ff966e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.204635 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.204898 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204636 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-sys\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.204898 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204665 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.204898 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204735 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cg8r\" (UniqueName: \"kubernetes.io/projected/c841b114-f106-47ab-a637-850293ff966e-kube-api-access-6cg8r\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.204898 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204788 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-root\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.204898 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204821 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfz6g\" (UniqueName: \"kubernetes.io/projected/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-kube-api-access-mfz6g\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.204898 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1021c7e-8fa9-4dac-b2be-17baa7deb980-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.205132 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.204899 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.305809 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.305778 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c841b114-f106-47ab-a637-850293ff966e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.305809 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.305812 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c841b114-f106-47ab-a637-850293ff966e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.306081 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.305831 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.306081 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.305855 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-sys\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306081 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.305885 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.306081 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.305981 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-sys\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306081 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306046 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6cg8r\" (UniqueName: \"kubernetes.io/projected/c841b114-f106-47ab-a637-850293ff966e-kube-api-access-6cg8r\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.306345 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306084 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-root\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306345 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306117 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mfz6g\" (UniqueName: \"kubernetes.io/projected/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-kube-api-access-mfz6g\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306345 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306142 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1021c7e-8fa9-4dac-b2be-17baa7deb980-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.306345 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306170 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306345 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306207 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-metrics-client-ca\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306345 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306238 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-tls\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306345 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306266 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306345 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306332 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.306882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306362 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-textfile\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306387 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c841b114-f106-47ab-a637-850293ff966e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.306882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306412 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-wtmp\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.306882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306452 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n6vk5\" (UniqueName: \"kubernetes.io/projected/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-api-access-n6vk5\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.306882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306484 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1021c7e-8fa9-4dac-b2be-17baa7deb980-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.306882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306799 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1021c7e-8fa9-4dac-b2be-17baa7deb980-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.306882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.306800 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.308785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.307940 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1021c7e-8fa9-4dac-b2be-17baa7deb980-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.308785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.308053 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-root\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.308785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.308084 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c841b114-f106-47ab-a637-850293ff966e-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.308785 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:27:20.308191 2561 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 02 05:27:20.308785 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:27:20.308260 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-tls podName:a1021c7e-8fa9-4dac-b2be-17baa7deb980 nodeName:}" failed. No retries permitted until 2026-06-02 05:27:20.80824026 +0000 UTC m=+92.518689486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-tls") pod "kube-state-metrics-8469c486bb-dvbh4" (UID: "a1021c7e-8fa9-4dac-b2be-17baa7deb980") : secret "kube-state-metrics-tls" not found Jun 02 05:27:20.308785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.308546 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-textfile\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.308785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.308552 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-wtmp\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.309236 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.309059 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.309236 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.309134 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c841b114-f106-47ab-a637-850293ff966e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.309236 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.309142 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-metrics-client-ca\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.309610 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.309587 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.309707 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.309611 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-tls\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.309821 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.309800 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c841b114-f106-47ab-a637-850293ff966e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.311189 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.311169 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.324101 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.324076 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6vk5\" (UniqueName: \"kubernetes.io/projected/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-api-access-n6vk5\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.324208 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.324162 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cg8r\" (UniqueName: \"kubernetes.io/projected/c841b114-f106-47ab-a637-850293ff966e-kube-api-access-6cg8r\") pod \"openshift-state-metrics-85658796db-7cfhl\" (UID: \"c841b114-f106-47ab-a637-850293ff966e\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.324617 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.324596 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfz6g\" (UniqueName: \"kubernetes.io/projected/f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e-kube-api-access-mfz6g\") pod \"node-exporter-rnmxr\" (UID: \"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e\") " pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.379247 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.379220 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" Jun 02 05:27:20.416019 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.415987 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rnmxr" Jun 02 05:27:20.428346 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:27:20.428087 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf60ae33c_91c4_40f5_9c7e_99e43fbc8a7e.slice/crio-e3853d614be9e47cf2d8a4f5b446c5afc6e50eeb2254bd4d7e57770dd9058bdc WatchSource:0}: Error finding container e3853d614be9e47cf2d8a4f5b446c5afc6e50eeb2254bd4d7e57770dd9058bdc: Status 404 returned error can't find the container with id e3853d614be9e47cf2d8a4f5b446c5afc6e50eeb2254bd4d7e57770dd9058bdc Jun 02 05:27:20.517991 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.517937 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-7cfhl"] Jun 02 05:27:20.521906 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:27:20.521878 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc841b114_f106_47ab_a637_850293ff966e.slice/crio-d3646637a10b4c0a63e030880ab9d8c1c01b217f8d8c161288e144e5f0b358bd WatchSource:0}: Error finding container d3646637a10b4c0a63e030880ab9d8c1c01b217f8d8c161288e144e5f0b358bd: Status 404 returned error can't find the container with id d3646637a10b4c0a63e030880ab9d8c1c01b217f8d8c161288e144e5f0b358bd Jun 02 05:27:20.810610 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.810538 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.812915 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.812894 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1021c7e-8fa9-4dac-b2be-17baa7deb980-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-dvbh4\" (UID: \"a1021c7e-8fa9-4dac-b2be-17baa7deb980\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:20.993656 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:20.993626 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" Jun 02 05:27:21.145836 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:21.145783 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rnmxr" event={"ID":"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e","Type":"ContainerStarted","Data":"e3853d614be9e47cf2d8a4f5b446c5afc6e50eeb2254bd4d7e57770dd9058bdc"} Jun 02 05:27:21.147743 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:21.147717 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4"] Jun 02 05:27:21.149548 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:21.149523 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" event={"ID":"c841b114-f106-47ab-a637-850293ff966e","Type":"ContainerStarted","Data":"2a6eb2abf55158e667f1a013cf11280413cd12e6f633fc561d427e722a1c5562"} Jun 02 05:27:21.149648 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:21.149553 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" event={"ID":"c841b114-f106-47ab-a637-850293ff966e","Type":"ContainerStarted","Data":"7a9a44034d338c091dc8a8c23cf48274752b46773bf250066e2ddeefa7bdaa93"} Jun 02 05:27:21.149648 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:21.149564 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" event={"ID":"c841b114-f106-47ab-a637-850293ff966e","Type":"ContainerStarted","Data":"d3646637a10b4c0a63e030880ab9d8c1c01b217f8d8c161288e144e5f0b358bd"} Jun 02 05:27:21.152835 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:27:21.152807 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1021c7e_8fa9_4dac_b2be_17baa7deb980.slice/crio-8006315d05771f73b22b428c9e892e6ab6eddbf287597d41bf9079fee4fc71c9 WatchSource:0}: Error finding container 8006315d05771f73b22b428c9e892e6ab6eddbf287597d41bf9079fee4fc71c9: Status 404 returned error can't find the container with id 8006315d05771f73b22b428c9e892e6ab6eddbf287597d41bf9079fee4fc71c9 Jun 02 05:27:22.153833 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:22.153801 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" event={"ID":"a1021c7e-8fa9-4dac-b2be-17baa7deb980","Type":"ContainerStarted","Data":"8006315d05771f73b22b428c9e892e6ab6eddbf287597d41bf9079fee4fc71c9"} Jun 02 05:27:23.158721 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:23.158674 2561 generic.go:358] "Generic (PLEG): container finished" podID="f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e" containerID="ab394990475e5f56a685b291edcec17a9d1bff3a23cd7f14c4e782dac56f986f" exitCode=0 Jun 02 05:27:23.159213 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:23.158766 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rnmxr" event={"ID":"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e","Type":"ContainerDied","Data":"ab394990475e5f56a685b291edcec17a9d1bff3a23cd7f14c4e782dac56f986f"} Jun 02 05:27:23.161543 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:23.161515 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" event={"ID":"c841b114-f106-47ab-a637-850293ff966e","Type":"ContainerStarted","Data":"b6003586b007cfd7e03597198d1fcc7a43bd4aa7afa7cbf33d074b1de454434c"} Jun 02 05:27:23.202126 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:23.200823 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-7cfhl" podStartSLOduration=2.530504059 podStartE2EDuration="4.200804624s" podCreationTimestamp="2026-06-02 05:27:19 +0000 UTC" firstStartedPulling="2026-06-02 05:27:20.784274808 +0000 UTC m=+92.494724032" lastFinishedPulling="2026-06-02 05:27:22.454575361 +0000 UTC m=+94.165024597" observedRunningTime="2026-06-02 05:27:23.200248995 +0000 UTC m=+94.910698241" watchObservedRunningTime="2026-06-02 05:27:23.200804624 +0000 UTC m=+94.911253884" Jun 02 05:27:24.167314 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:24.167269 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" event={"ID":"a1021c7e-8fa9-4dac-b2be-17baa7deb980","Type":"ContainerStarted","Data":"754ca1c991fb8a953e3bb0a67f0e0d7fcf0d7f0b5442b09b348f6c91c703693b"} Jun 02 05:27:24.167314 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:24.167319 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" event={"ID":"a1021c7e-8fa9-4dac-b2be-17baa7deb980","Type":"ContainerStarted","Data":"380959759bf8d763453b3e5d90610af37651a9cee8ca22d3c4b9e4f7b29b2f53"} Jun 02 05:27:24.167849 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:24.167332 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" event={"ID":"a1021c7e-8fa9-4dac-b2be-17baa7deb980","Type":"ContainerStarted","Data":"9b6b75279ed4d4dd8f43c74cd59bb70ff9092c4fa47f11faadba35c42ba28a67"} Jun 02 05:27:24.169195 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:24.169170 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rnmxr" event={"ID":"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e","Type":"ContainerStarted","Data":"6bd61937b0ddb1c022125e093b87c5bd5c598d27798894055f3176fabc46f102"} Jun 02 05:27:24.169311 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:24.169206 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rnmxr" event={"ID":"f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e","Type":"ContainerStarted","Data":"996c95a2a467ba71eb655dd4d41f2dab46c9f73d1222cca26802dfe81f95c577"} Jun 02 05:27:24.189244 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:24.189190 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-dvbh4" podStartSLOduration=1.607941208 podStartE2EDuration="4.189176005s" podCreationTimestamp="2026-06-02 05:27:20 +0000 UTC" firstStartedPulling="2026-06-02 05:27:21.154888773 +0000 UTC m=+92.865338003" lastFinishedPulling="2026-06-02 05:27:23.736123576 +0000 UTC m=+95.446572800" observedRunningTime="2026-06-02 05:27:24.18833068 +0000 UTC m=+95.898779923" watchObservedRunningTime="2026-06-02 05:27:24.189176005 +0000 UTC m=+95.899625284" Jun 02 05:27:24.210327 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:24.210268 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rnmxr" podStartSLOduration=2.187660768 podStartE2EDuration="4.210254252s" podCreationTimestamp="2026-06-02 05:27:20 +0000 UTC" firstStartedPulling="2026-06-02 05:27:20.429541547 +0000 UTC m=+92.139990771" lastFinishedPulling="2026-06-02 05:27:22.452135017 +0000 UTC m=+94.162584255" observedRunningTime="2026-06-02 05:27:24.209576228 +0000 UTC m=+95.920025486" watchObservedRunningTime="2026-06-02 05:27:24.210254252 +0000 UTC m=+95.920703498" Jun 02 05:27:26.410588 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.410539 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 05:27:26.435919 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.435887 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.461000 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.460953 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 05:27:26.461633 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.461605 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 05:27:26.462123 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.462090 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gsb26\"" Jun 02 05:27:26.462123 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.462097 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 05:27:26.462336 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.462206 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 05:27:26.462399 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.462385 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 05:27:26.462471 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.462420 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-29c8bblvvdgpa\"" Jun 02 05:27:26.462471 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.462431 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 05:27:26.462732 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.462715 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 05:27:26.462786 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.462732 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 05:27:26.463138 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.463105 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 05:27:26.463138 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.463118 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 05:27:26.463356 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.463341 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 05:27:26.464832 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.464815 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 05:27:26.475002 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.474983 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 05:27:26.475785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.475771 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 05:27:26.559511 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559475 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559683 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559529 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559683 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559584 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559683 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559625 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559683 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559855 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559701 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559855 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559743 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559855 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559762 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559855 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559777 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-config\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559855 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559793 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559855 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559808 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knk55\" (UniqueName: \"kubernetes.io/projected/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-kube-api-access-knk55\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.559855 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.560202 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559873 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.560202 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559934 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.560202 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.559987 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.560202 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.560015 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.560202 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.560058 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.560202 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.560084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.660659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660572 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.660659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660608 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.660659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660627 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.660659 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660663 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660691 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660724 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660774 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660804 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660838 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660867 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.660889 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661395 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661243 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661395 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661280 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661395 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661308 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-config\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661395 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661333 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661395 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661356 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-knk55\" (UniqueName: \"kubernetes.io/projected/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-kube-api-access-knk55\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661395 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661383 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661846 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661416 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661846 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661469 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.661846 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661756 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.662025 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.661871 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.664499 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.664470 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.664594 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.664532 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.664668 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.664648 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.664705 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.664693 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.665051 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.665024 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.665164 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.665054 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.665751 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.665635 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.666359 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.665850 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.666359 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.665987 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.666359 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.666139 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-config\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.666359 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.666302 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.667207 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.667189 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.667518 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.667498 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.667644 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.667627 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.672560 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.672543 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knk55\" (UniqueName: \"kubernetes.io/projected/2d35c990-be4a-4fe8-a01c-8f015ca3e99c-kube-api-access-knk55\") pod \"prometheus-k8s-0\" (UID: \"2d35c990-be4a-4fe8-a01c-8f015ca3e99c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.744588 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.744545 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:26.884384 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:26.884033 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 05:27:27.179362 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:27.179322 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d35c990-be4a-4fe8-a01c-8f015ca3e99c","Type":"ContainerStarted","Data":"62ae27b9a341c5f83d67c256336604051befe61f4eeb32b3b874080c00d745f4"} Jun 02 05:27:29.186061 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:29.186023 2561 generic.go:358] "Generic (PLEG): container finished" podID="2d35c990-be4a-4fe8-a01c-8f015ca3e99c" containerID="9324ba9d05a394ece3c9071fc1845abd716dfeb18531a584ba50303697768bff" exitCode=0 Jun 02 05:27:29.186433 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:29.186111 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d35c990-be4a-4fe8-a01c-8f015ca3e99c","Type":"ContainerDied","Data":"9324ba9d05a394ece3c9071fc1845abd716dfeb18531a584ba50303697768bff"} Jun 02 05:27:30.720562 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:30.720488 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5d76657986-fp4z5" podUID="2b70cc86-eebd-4e20-961b-743fead30181" containerName="console" containerID="cri-o://57863195339ef9e8e84b7d46078afc1730ab21690293d9384a14c7154d57915c" gracePeriod=15 Jun 02 05:27:31.072805 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:31.072773 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9vkts" Jun 02 05:27:31.196564 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:31.196533 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d76657986-fp4z5_2b70cc86-eebd-4e20-961b-743fead30181/console/0.log" Jun 02 05:27:31.196729 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:31.196586 2561 generic.go:358] "Generic (PLEG): container finished" podID="2b70cc86-eebd-4e20-961b-743fead30181" containerID="57863195339ef9e8e84b7d46078afc1730ab21690293d9384a14c7154d57915c" exitCode=2 Jun 02 05:27:31.196729 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:31.196643 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d76657986-fp4z5" event={"ID":"2b70cc86-eebd-4e20-961b-743fead30181","Type":"ContainerDied","Data":"57863195339ef9e8e84b7d46078afc1730ab21690293d9384a14c7154d57915c"} Jun 02 05:27:31.953799 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:31.953779 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d76657986-fp4z5_2b70cc86-eebd-4e20-961b-743fead30181/console/0.log" Jun 02 05:27:31.954076 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:31.953839 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:27:32.010791 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.010759 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4kmp\" (UniqueName: \"kubernetes.io/projected/2b70cc86-eebd-4e20-961b-743fead30181-kube-api-access-p4kmp\") pod \"2b70cc86-eebd-4e20-961b-743fead30181\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " Jun 02 05:27:32.010791 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.010795 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-service-ca\") pod \"2b70cc86-eebd-4e20-961b-743fead30181\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " Jun 02 05:27:32.011060 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.010822 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-serving-cert\") pod \"2b70cc86-eebd-4e20-961b-743fead30181\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " Jun 02 05:27:32.011060 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.010862 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-oauth-serving-cert\") pod \"2b70cc86-eebd-4e20-961b-743fead30181\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " Jun 02 05:27:32.011060 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.010881 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-oauth-config\") pod \"2b70cc86-eebd-4e20-961b-743fead30181\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " Jun 02 05:27:32.011060 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.010928 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-console-config\") pod \"2b70cc86-eebd-4e20-961b-743fead30181\" (UID: \"2b70cc86-eebd-4e20-961b-743fead30181\") " Jun 02 05:27:32.011255 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.011206 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-service-ca" (OuterVolumeSpecName: "service-ca") pod "2b70cc86-eebd-4e20-961b-743fead30181" (UID: "2b70cc86-eebd-4e20-961b-743fead30181"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 05:27:32.011386 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.011363 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2b70cc86-eebd-4e20-961b-743fead30181" (UID: "2b70cc86-eebd-4e20-961b-743fead30181"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 05:27:32.011440 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.011405 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-console-config" (OuterVolumeSpecName: "console-config") pod "2b70cc86-eebd-4e20-961b-743fead30181" (UID: "2b70cc86-eebd-4e20-961b-743fead30181"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 05:27:32.013344 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.013315 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2b70cc86-eebd-4e20-961b-743fead30181" (UID: "2b70cc86-eebd-4e20-961b-743fead30181"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 05:27:32.013344 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.013329 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b70cc86-eebd-4e20-961b-743fead30181-kube-api-access-p4kmp" (OuterVolumeSpecName: "kube-api-access-p4kmp") pod "2b70cc86-eebd-4e20-961b-743fead30181" (UID: "2b70cc86-eebd-4e20-961b-743fead30181"). InnerVolumeSpecName "kube-api-access-p4kmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:27:32.013480 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.013340 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2b70cc86-eebd-4e20-961b-743fead30181" (UID: "2b70cc86-eebd-4e20-961b-743fead30181"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 05:27:32.112237 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.112199 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4kmp\" (UniqueName: \"kubernetes.io/projected/2b70cc86-eebd-4e20-961b-743fead30181-kube-api-access-p4kmp\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:32.112237 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.112230 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-service-ca\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:32.112237 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.112243 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-serving-cert\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:32.112461 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.112257 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-oauth-serving-cert\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:32.112461 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.112269 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b70cc86-eebd-4e20-961b-743fead30181-console-oauth-config\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:32.112461 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.112280 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b70cc86-eebd-4e20-961b-743fead30181-console-config\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:27:32.200415 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.200340 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d76657986-fp4z5_2b70cc86-eebd-4e20-961b-743fead30181/console/0.log" Jun 02 05:27:32.200582 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.200438 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d76657986-fp4z5" event={"ID":"2b70cc86-eebd-4e20-961b-743fead30181","Type":"ContainerDied","Data":"823f08c1e1d79b3ce43771523bef0b5bbe8565e44f3e5651660d08207ba1f7ea"} Jun 02 05:27:32.200582 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.200469 2561 scope.go:117] "RemoveContainer" containerID="57863195339ef9e8e84b7d46078afc1730ab21690293d9384a14c7154d57915c" Jun 02 05:27:32.200582 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.200467 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d76657986-fp4z5" Jun 02 05:27:32.222392 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.222364 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d76657986-fp4z5"] Jun 02 05:27:32.227829 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.227805 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d76657986-fp4z5"] Jun 02 05:27:32.785630 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:32.785557 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b70cc86-eebd-4e20-961b-743fead30181" path="/var/lib/kubelet/pods/2b70cc86-eebd-4e20-961b-743fead30181/volumes" Jun 02 05:27:33.205705 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:33.205668 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d35c990-be4a-4fe8-a01c-8f015ca3e99c","Type":"ContainerStarted","Data":"fa69f892c875ea990b64f6d6a828f0c87466afabe2415cac24569acdfeca209b"} Jun 02 05:27:33.206214 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:33.205711 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d35c990-be4a-4fe8-a01c-8f015ca3e99c","Type":"ContainerStarted","Data":"f2269f7213d373390cc0c5a81b64733b09545792a6969e46dfc14cbfd11f1157"} Jun 02 05:27:35.215162 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:35.215130 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d35c990-be4a-4fe8-a01c-8f015ca3e99c","Type":"ContainerStarted","Data":"b39b0c472f4aeefa0134f2006958dc565ccc00a749c610cd1b36039266d35f4d"} Jun 02 05:27:35.215162 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:35.215166 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d35c990-be4a-4fe8-a01c-8f015ca3e99c","Type":"ContainerStarted","Data":"720536fea58dcb8584a57d4ce14ce191523452d479d163a023528d203a4beb90"} Jun 02 05:27:35.215621 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:35.215176 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d35c990-be4a-4fe8-a01c-8f015ca3e99c","Type":"ContainerStarted","Data":"61c02de91b93fb8f410994109d56306d47e85419e50bbd0c8653a6dd032a24b1"} Jun 02 05:27:35.215621 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:35.215185 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d35c990-be4a-4fe8-a01c-8f015ca3e99c","Type":"ContainerStarted","Data":"3c345d8cec523bc4dd39cd47b945b400f051b3795860e26f9ece12d6d489bdb2"} Jun 02 05:27:35.247491 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:35.247373 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.61468955 podStartE2EDuration="9.247356968s" podCreationTimestamp="2026-06-02 05:27:26 +0000 UTC" firstStartedPulling="2026-06-02 05:27:26.89024905 +0000 UTC m=+98.600698278" lastFinishedPulling="2026-06-02 05:27:34.52291647 +0000 UTC m=+106.233365696" observedRunningTime="2026-06-02 05:27:35.24688384 +0000 UTC m=+106.957333086" watchObservedRunningTime="2026-06-02 05:27:35.247356968 +0000 UTC m=+106.957806215" Jun 02 05:27:36.745600 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:36.745568 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:27:40.811440 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:27:40.811401 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rw625_7c31483d-0694-44d8-9125-f41ddf67e3d9/serve-healthcheck-canary/0.log" Jun 02 05:28:26.745179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:28:26.745141 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:28:26.765374 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:28:26.765349 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:28:27.372734 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:28:27.372707 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 05:30:48.673605 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:30:48.673578 2561 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 05:31:11.710844 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.710806 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4"] Jun 02 05:31:11.713159 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.711086 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b70cc86-eebd-4e20-961b-743fead30181" containerName="console" Jun 02 05:31:11.713159 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.711097 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b70cc86-eebd-4e20-961b-743fead30181" containerName="console" Jun 02 05:31:11.713159 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.711144 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b70cc86-eebd-4e20-961b-743fead30181" containerName="console" Jun 02 05:31:11.714056 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.714041 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.716589 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.716565 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 05:31:11.716711 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.716569 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 05:31:11.716711 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.716572 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 05:31:11.716711 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.716690 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8rj7j\"" Jun 02 05:31:11.716947 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.716904 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 05:31:11.728127 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.728103 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4"] Jun 02 05:31:11.861520 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.861485 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-tmp\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.861694 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.861548 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msjwp\" (UniqueName: \"kubernetes.io/projected/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-kube-api-access-msjwp\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.861694 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.861586 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-manifests\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.861694 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.861606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.861694 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.861636 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.962907 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.962785 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-tmp\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.963130 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.962906 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-msjwp\" (UniqueName: \"kubernetes.io/projected/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-kube-api-access-msjwp\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.963130 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.962983 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-manifests\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.963252 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.963179 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.963252 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.963246 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.963385 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.963347 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-manifests\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.965504 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.965476 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-tmp\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.966123 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.966103 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.966286 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.966268 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:11.971198 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:11.971177 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msjwp\" (UniqueName: \"kubernetes.io/projected/7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d-kube-api-access-msjwp\") pod \"opendatahub-operator-controller-manager-59f99d6f45-5crx4\" (UID: \"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d\") " pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:12.023853 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:12.023825 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:12.191731 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:12.191702 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4"] Jun 02 05:31:12.194995 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:31:12.194945 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb4bc5f_d0e1_4a99_ae80_9f773d45d55d.slice/crio-39a5dce1c61396ff83a3d3ba7666834aca9e6d080211eefd158ff4af7111da15 WatchSource:0}: Error finding container 39a5dce1c61396ff83a3d3ba7666834aca9e6d080211eefd158ff4af7111da15: Status 404 returned error can't find the container with id 39a5dce1c61396ff83a3d3ba7666834aca9e6d080211eefd158ff4af7111da15 Jun 02 05:31:12.197268 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:12.197249 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 05:31:12.790567 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:12.790530 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" event={"ID":"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d","Type":"ContainerStarted","Data":"39a5dce1c61396ff83a3d3ba7666834aca9e6d080211eefd158ff4af7111da15"} Jun 02 05:31:15.801818 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:15.801780 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" event={"ID":"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d","Type":"ContainerStarted","Data":"7d11f93ddb8c8ed2c2116e033373bbd6790996eddbdccee41365a3dc22341681"} Jun 02 05:31:16.805063 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:16.805031 2561 generic.go:358] "Generic (PLEG): container finished" podID="7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d" containerID="7d11f93ddb8c8ed2c2116e033373bbd6790996eddbdccee41365a3dc22341681" exitCode=0 Jun 02 05:31:16.805427 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:16.805116 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" event={"ID":"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d","Type":"ContainerDied","Data":"7d11f93ddb8c8ed2c2116e033373bbd6790996eddbdccee41365a3dc22341681"} Jun 02 05:31:17.808878 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:17.808847 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" event={"ID":"7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d","Type":"ContainerStarted","Data":"c3e781eab7824a6c8958915b7b62b4a22560a5e8dba28781d68fb91ebe3e7deb"} Jun 02 05:31:17.809244 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:17.808945 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:17.830815 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:17.830774 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" podStartSLOduration=1.8181102839999999 podStartE2EDuration="6.830761856s" podCreationTimestamp="2026-06-02 05:31:11 +0000 UTC" firstStartedPulling="2026-06-02 05:31:12.197407776 +0000 UTC m=+323.907857000" lastFinishedPulling="2026-06-02 05:31:17.210059339 +0000 UTC m=+328.920508572" observedRunningTime="2026-06-02 05:31:17.828917206 +0000 UTC m=+329.539366452" watchObservedRunningTime="2026-06-02 05:31:17.830761856 +0000 UTC m=+329.541211167" Jun 02 05:31:28.813941 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:28.813914 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59f99d6f45-5crx4" Jun 02 05:31:30.299697 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.299660 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk"] Jun 02 05:31:30.311951 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.311919 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.314821 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.314790 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 05:31:30.314939 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.314831 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 05:31:30.316023 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.316002 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bjnr6\"" Jun 02 05:31:30.316148 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.316065 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 05:31:30.316216 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.316156 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 05:31:30.316281 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.316216 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 05:31:30.316536 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.316514 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk"] Jun 02 05:31:30.408291 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.408258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b9308d4e-f473-4244-9583-a2a855743679-manager-config\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.408291 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.408294 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65x7n\" (UniqueName: \"kubernetes.io/projected/b9308d4e-f473-4244-9583-a2a855743679-kube-api-access-65x7n\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.408524 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.408334 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9308d4e-f473-4244-9583-a2a855743679-cert\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.408524 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.408430 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9308d4e-f473-4244-9583-a2a855743679-metrics-cert\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.508953 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.508923 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-65x7n\" (UniqueName: \"kubernetes.io/projected/b9308d4e-f473-4244-9583-a2a855743679-kube-api-access-65x7n\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.509121 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.509002 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9308d4e-f473-4244-9583-a2a855743679-cert\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.509121 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.509035 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9308d4e-f473-4244-9583-a2a855743679-metrics-cert\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.509121 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.509059 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b9308d4e-f473-4244-9583-a2a855743679-manager-config\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.509747 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.509721 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b9308d4e-f473-4244-9583-a2a855743679-manager-config\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.511671 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.511645 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9308d4e-f473-4244-9583-a2a855743679-metrics-cert\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.511838 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.511821 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9308d4e-f473-4244-9583-a2a855743679-cert\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.549785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.549728 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65x7n\" (UniqueName: \"kubernetes.io/projected/b9308d4e-f473-4244-9583-a2a855743679-kube-api-access-65x7n\") pod \"lws-controller-manager-647f887c64-pcfzk\" (UID: \"b9308d4e-f473-4244-9583-a2a855743679\") " pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.622012 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.621980 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:30.756583 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.756545 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk"] Jun 02 05:31:30.759268 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:31:30.759232 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9308d4e_f473_4244_9583_a2a855743679.slice/crio-3bc74e4b6d9b2ace8942c1e9be319e6670ca9f479d8ca7b7a0570556646deb9e WatchSource:0}: Error finding container 3bc74e4b6d9b2ace8942c1e9be319e6670ca9f479d8ca7b7a0570556646deb9e: Status 404 returned error can't find the container with id 3bc74e4b6d9b2ace8942c1e9be319e6670ca9f479d8ca7b7a0570556646deb9e Jun 02 05:31:30.839580 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:30.839504 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" event={"ID":"b9308d4e-f473-4244-9583-a2a855743679","Type":"ContainerStarted","Data":"3bc74e4b6d9b2ace8942c1e9be319e6670ca9f479d8ca7b7a0570556646deb9e"} Jun 02 05:31:31.651943 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.651907 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp"] Jun 02 05:31:31.656056 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.656032 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.658552 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.658525 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 05:31:31.658692 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.658578 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 05:31:31.659530 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.659509 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 05:31:31.659638 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.659551 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 05:31:31.659638 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.659590 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4qm5l\"" Jun 02 05:31:31.671346 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.671307 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp"] Jun 02 05:31:31.717767 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.717726 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95e46288-fece-4e36-9b2b-b801570e76ab-tls-certs\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.717767 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.717768 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxbpd\" (UniqueName: \"kubernetes.io/projected/95e46288-fece-4e36-9b2b-b801570e76ab-kube-api-access-jxbpd\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.718027 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.717804 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95e46288-fece-4e36-9b2b-b801570e76ab-tmp\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.819160 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.818790 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95e46288-fece-4e36-9b2b-b801570e76ab-tls-certs\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.819160 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.818835 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jxbpd\" (UniqueName: \"kubernetes.io/projected/95e46288-fece-4e36-9b2b-b801570e76ab-kube-api-access-jxbpd\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.819160 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.818867 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95e46288-fece-4e36-9b2b-b801570e76ab-tmp\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.822013 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.821940 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95e46288-fece-4e36-9b2b-b801570e76ab-tls-certs\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.822346 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.822309 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95e46288-fece-4e36-9b2b-b801570e76ab-tmp\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.829047 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.829002 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxbpd\" (UniqueName: \"kubernetes.io/projected/95e46288-fece-4e36-9b2b-b801570e76ab-kube-api-access-jxbpd\") pod \"kube-auth-proxy-6c78f6699c-l4rxp\" (UID: \"95e46288-fece-4e36-9b2b-b801570e76ab\") " pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:31.970830 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:31.970749 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" Jun 02 05:31:32.117646 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:32.117615 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp"] Jun 02 05:31:32.121733 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:31:32.121702 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e46288_fece_4e36_9b2b_b801570e76ab.slice/crio-740f44db198ac4c2f283263df3d9527eb6e7dce38e93ad6e00219faaabd747ec WatchSource:0}: Error finding container 740f44db198ac4c2f283263df3d9527eb6e7dce38e93ad6e00219faaabd747ec: Status 404 returned error can't find the container with id 740f44db198ac4c2f283263df3d9527eb6e7dce38e93ad6e00219faaabd747ec Jun 02 05:31:32.847076 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:32.847034 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" event={"ID":"95e46288-fece-4e36-9b2b-b801570e76ab","Type":"ContainerStarted","Data":"740f44db198ac4c2f283263df3d9527eb6e7dce38e93ad6e00219faaabd747ec"} Jun 02 05:31:33.852371 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:33.852324 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" event={"ID":"b9308d4e-f473-4244-9583-a2a855743679","Type":"ContainerStarted","Data":"8288b7acfc448eea37a445150cf2e8cbff6f8be958951d284fb64391d727810a"} Jun 02 05:31:33.852859 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:33.852401 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:31:33.877917 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:33.877852 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" podStartSLOduration=1.310105303 podStartE2EDuration="3.877834825s" podCreationTimestamp="2026-06-02 05:31:30 +0000 UTC" firstStartedPulling="2026-06-02 05:31:30.761134307 +0000 UTC m=+342.471583533" lastFinishedPulling="2026-06-02 05:31:33.328863829 +0000 UTC m=+345.039313055" observedRunningTime="2026-06-02 05:31:33.875692319 +0000 UTC m=+345.586141566" watchObservedRunningTime="2026-06-02 05:31:33.877834825 +0000 UTC m=+345.588284071" Jun 02 05:31:35.859166 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:35.859126 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" event={"ID":"95e46288-fece-4e36-9b2b-b801570e76ab","Type":"ContainerStarted","Data":"6ba0254c522af51ef234a84268816fcc55769c7e2bbe8a5d89a8badda5ea1d4e"} Jun 02 05:31:35.878263 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:35.878215 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c78f6699c-l4rxp" podStartSLOduration=1.261373973 podStartE2EDuration="4.87819995s" podCreationTimestamp="2026-06-02 05:31:31 +0000 UTC" firstStartedPulling="2026-06-02 05:31:32.124033932 +0000 UTC m=+343.834483156" lastFinishedPulling="2026-06-02 05:31:35.740859893 +0000 UTC m=+347.451309133" observedRunningTime="2026-06-02 05:31:35.876456787 +0000 UTC m=+347.586906035" watchObservedRunningTime="2026-06-02 05:31:35.87819995 +0000 UTC m=+347.588649197" Jun 02 05:31:44.858290 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:31:44.858259 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-647f887c64-pcfzk" Jun 02 05:32:07.221823 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.221784 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz"] Jun 02 05:32:07.231722 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.231691 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" Jun 02 05:32:07.235165 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.235134 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 02 05:32:07.235309 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.235247 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rgprj\"" Jun 02 05:32:07.237872 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.237846 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz"] Jun 02 05:32:07.418278 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.418240 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/6e9d48c4-d63c-47a9-84be-966dbadca366-redirect-config\") pod \"dashboard-redirect-7f9b65cdbd-mxjmz\" (UID: \"6e9d48c4-d63c-47a9-84be-966dbadca366\") " pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" Jun 02 05:32:07.418437 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.418302 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrjxc\" (UniqueName: \"kubernetes.io/projected/6e9d48c4-d63c-47a9-84be-966dbadca366-kube-api-access-mrjxc\") pod \"dashboard-redirect-7f9b65cdbd-mxjmz\" (UID: \"6e9d48c4-d63c-47a9-84be-966dbadca366\") " pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" Jun 02 05:32:07.519157 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.519082 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/6e9d48c4-d63c-47a9-84be-966dbadca366-redirect-config\") pod \"dashboard-redirect-7f9b65cdbd-mxjmz\" (UID: \"6e9d48c4-d63c-47a9-84be-966dbadca366\") " pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" Jun 02 05:32:07.519157 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.519136 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mrjxc\" (UniqueName: \"kubernetes.io/projected/6e9d48c4-d63c-47a9-84be-966dbadca366-kube-api-access-mrjxc\") pod \"dashboard-redirect-7f9b65cdbd-mxjmz\" (UID: \"6e9d48c4-d63c-47a9-84be-966dbadca366\") " pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" Jun 02 05:32:07.519758 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.519739 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/6e9d48c4-d63c-47a9-84be-966dbadca366-redirect-config\") pod \"dashboard-redirect-7f9b65cdbd-mxjmz\" (UID: \"6e9d48c4-d63c-47a9-84be-966dbadca366\") " pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" Jun 02 05:32:07.534946 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.534924 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrjxc\" (UniqueName: \"kubernetes.io/projected/6e9d48c4-d63c-47a9-84be-966dbadca366-kube-api-access-mrjxc\") pod \"dashboard-redirect-7f9b65cdbd-mxjmz\" (UID: \"6e9d48c4-d63c-47a9-84be-966dbadca366\") " pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" Jun 02 05:32:07.542764 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.542727 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" Jun 02 05:32:07.668782 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.668758 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz"] Jun 02 05:32:07.671366 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:32:07.671330 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9d48c4_d63c_47a9_84be_966dbadca366.slice/crio-94f2ffa6a4a0cb7587595fb15159bf3e405db2dabbbe7df879ca0166ca6d9a54 WatchSource:0}: Error finding container 94f2ffa6a4a0cb7587595fb15159bf3e405db2dabbbe7df879ca0166ca6d9a54: Status 404 returned error can't find the container with id 94f2ffa6a4a0cb7587595fb15159bf3e405db2dabbbe7df879ca0166ca6d9a54 Jun 02 05:32:07.946731 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:07.946696 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" event={"ID":"6e9d48c4-d63c-47a9-84be-966dbadca366","Type":"ContainerStarted","Data":"94f2ffa6a4a0cb7587595fb15159bf3e405db2dabbbe7df879ca0166ca6d9a54"} Jun 02 05:32:11.958578 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:11.958543 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" event={"ID":"6e9d48c4-d63c-47a9-84be-966dbadca366","Type":"ContainerStarted","Data":"a49fe4e2c06497ae92ce2c3417aa6fa64afe0bcfc3977545d43eea9c8701d14c"} Jun 02 05:32:11.978680 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:32:11.978630 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-7f9b65cdbd-mxjmz" podStartSLOduration=1.584280359 podStartE2EDuration="4.978615402s" podCreationTimestamp="2026-06-02 05:32:07 +0000 UTC" firstStartedPulling="2026-06-02 05:32:07.672990168 +0000 UTC m=+379.383439391" lastFinishedPulling="2026-06-02 05:32:11.06732519 +0000 UTC m=+382.777774434" observedRunningTime="2026-06-02 05:32:11.977204005 +0000 UTC m=+383.687653250" watchObservedRunningTime="2026-06-02 05:32:11.978615402 +0000 UTC m=+383.689064647" Jun 02 05:33:08.567660 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.567581 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q"] Jun 02 05:33:08.570970 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.570940 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:08.578069 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.578049 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 05:33:08.578212 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.578192 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kqvl8\"" Jun 02 05:33:08.578922 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.578907 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 05:33:08.586837 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.586810 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q"] Jun 02 05:33:08.628454 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.628428 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62lnq\" (UniqueName: \"kubernetes.io/projected/8c6f1be9-606c-435a-8dcb-c3fda0677e12-kube-api-access-62lnq\") pod \"limitador-operator-controller-manager-85c4996f8c-z6m6q\" (UID: \"8c6f1be9-606c-435a-8dcb-c3fda0677e12\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:08.729250 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.729223 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-62lnq\" (UniqueName: \"kubernetes.io/projected/8c6f1be9-606c-435a-8dcb-c3fda0677e12-kube-api-access-62lnq\") pod \"limitador-operator-controller-manager-85c4996f8c-z6m6q\" (UID: \"8c6f1be9-606c-435a-8dcb-c3fda0677e12\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:08.739729 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.739710 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62lnq\" (UniqueName: \"kubernetes.io/projected/8c6f1be9-606c-435a-8dcb-c3fda0677e12-kube-api-access-62lnq\") pod \"limitador-operator-controller-manager-85c4996f8c-z6m6q\" (UID: \"8c6f1be9-606c-435a-8dcb-c3fda0677e12\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:08.881426 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:08.881349 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:09.047746 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:09.047700 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q"] Jun 02 05:33:09.052178 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:33:09.052131 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6f1be9_606c_435a_8dcb_c3fda0677e12.slice/crio-6f1d5a623f1c9cd22dcab63a2fa8f14f193f6e56c215b53167f00e918f334c82 WatchSource:0}: Error finding container 6f1d5a623f1c9cd22dcab63a2fa8f14f193f6e56c215b53167f00e918f334c82: Status 404 returned error can't find the container with id 6f1d5a623f1c9cd22dcab63a2fa8f14f193f6e56c215b53167f00e918f334c82 Jun 02 05:33:09.126142 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:09.126106 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" event={"ID":"8c6f1be9-606c-435a-8dcb-c3fda0677e12","Type":"ContainerStarted","Data":"6f1d5a623f1c9cd22dcab63a2fa8f14f193f6e56c215b53167f00e918f334c82"} Jun 02 05:33:11.133553 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:11.133505 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" event={"ID":"8c6f1be9-606c-435a-8dcb-c3fda0677e12","Type":"ContainerStarted","Data":"ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37"} Jun 02 05:33:11.133928 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:11.133635 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:11.158058 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:11.157998 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" podStartSLOduration=1.170369909 podStartE2EDuration="3.157981865s" podCreationTimestamp="2026-06-02 05:33:08 +0000 UTC" firstStartedPulling="2026-06-02 05:33:09.054288899 +0000 UTC m=+440.764738122" lastFinishedPulling="2026-06-02 05:33:11.041900839 +0000 UTC m=+442.752350078" observedRunningTime="2026-06-02 05:33:11.154595766 +0000 UTC m=+442.865045012" watchObservedRunningTime="2026-06-02 05:33:11.157981865 +0000 UTC m=+442.868431106" Jun 02 05:33:15.045215 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.045184 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7"] Jun 02 05:33:15.048534 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.048515 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.051721 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.051704 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 02 05:33:15.052227 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.052207 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 02 05:33:15.052692 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.052674 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xgl64\"" Jun 02 05:33:15.062101 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.062080 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7"] Jun 02 05:33:15.081913 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.081889 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.082027 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.081931 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.082027 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.082000 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp7mb\" (UniqueName: \"kubernetes.io/projected/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-kube-api-access-hp7mb\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.182385 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.182359 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hp7mb\" (UniqueName: \"kubernetes.io/projected/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-kube-api-access-hp7mb\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.182556 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.182421 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.182556 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.182445 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.183067 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.183049 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.185221 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.185192 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.191316 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.191286 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp7mb\" (UniqueName: \"kubernetes.io/projected/1057b8a8-6d5d-4775-8b02-53e6e55f1e04-kube-api-access-hp7mb\") pod \"kuadrant-console-plugin-6cb54b5c86-55vg7\" (UID: \"1057b8a8-6d5d-4775-8b02-53e6e55f1e04\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.363888 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.363814 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" Jun 02 05:33:15.493209 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:15.493179 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7"] Jun 02 05:33:15.496402 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:33:15.496364 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1057b8a8_6d5d_4775_8b02_53e6e55f1e04.slice/crio-3ae9152d3b264c7923db6e20e4204cf61da762098ff1be550e0045756e4317fc WatchSource:0}: Error finding container 3ae9152d3b264c7923db6e20e4204cf61da762098ff1be550e0045756e4317fc: Status 404 returned error can't find the container with id 3ae9152d3b264c7923db6e20e4204cf61da762098ff1be550e0045756e4317fc Jun 02 05:33:16.150660 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:16.150627 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" event={"ID":"1057b8a8-6d5d-4775-8b02-53e6e55f1e04","Type":"ContainerStarted","Data":"3ae9152d3b264c7923db6e20e4204cf61da762098ff1be550e0045756e4317fc"} Jun 02 05:33:22.138507 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:22.138477 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:26.085172 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.085133 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q"] Jun 02 05:33:26.085621 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.085420 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" containerName="manager" containerID="cri-o://ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37" gracePeriod=2 Jun 02 05:33:26.095409 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.095371 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q"] Jun 02 05:33:26.162665 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.162629 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm"] Jun 02 05:33:26.163390 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.163092 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" containerName="manager" Jun 02 05:33:26.163390 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.163112 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" containerName="manager" Jun 02 05:33:26.163390 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.163200 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" containerName="manager" Jun 02 05:33:26.166701 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.166676 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" Jun 02 05:33:26.174095 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.174055 2561 status_manager.go:895] "Failed to get status for pod" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" err="pods \"limitador-operator-controller-manager-85c4996f8c-z6m6q\" is forbidden: User \"system:node:ip-10-0-141-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-22.ec2.internal' and this object" Jun 02 05:33:26.193522 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.193492 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm"] Jun 02 05:33:26.278681 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.278648 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz5p9\" (UniqueName: \"kubernetes.io/projected/ab3ac6ce-4fcf-4b32-b110-baf01995e4b7-kube-api-access-mz5p9\") pod \"limitador-operator-controller-manager-85c4996f8c-btkhm\" (UID: \"ab3ac6ce-4fcf-4b32-b110-baf01995e4b7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" Jun 02 05:33:26.379560 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.379471 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mz5p9\" (UniqueName: \"kubernetes.io/projected/ab3ac6ce-4fcf-4b32-b110-baf01995e4b7-kube-api-access-mz5p9\") pod \"limitador-operator-controller-manager-85c4996f8c-btkhm\" (UID: \"ab3ac6ce-4fcf-4b32-b110-baf01995e4b7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" Jun 02 05:33:26.394900 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.394867 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz5p9\" (UniqueName: \"kubernetes.io/projected/ab3ac6ce-4fcf-4b32-b110-baf01995e4b7-kube-api-access-mz5p9\") pod \"limitador-operator-controller-manager-85c4996f8c-btkhm\" (UID: \"ab3ac6ce-4fcf-4b32-b110-baf01995e4b7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" Jun 02 05:33:26.479844 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:26.479803 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" Jun 02 05:33:39.587376 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:39.587345 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm"] Jun 02 05:33:39.714819 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:33:39.714787 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3ac6ce_4fcf_4b32_b110_baf01995e4b7.slice/crio-b8c611a3d554380761e33b3fe31d33eb7bdbe0ad9e15b4ad0908ccf44dd00465 WatchSource:0}: Error finding container b8c611a3d554380761e33b3fe31d33eb7bdbe0ad9e15b4ad0908ccf44dd00465: Status 404 returned error can't find the container with id b8c611a3d554380761e33b3fe31d33eb7bdbe0ad9e15b4ad0908ccf44dd00465 Jun 02 05:33:39.732692 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:39.732675 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:39.734855 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:39.734829 2561 status_manager.go:895] "Failed to get status for pod" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" err="pods \"limitador-operator-controller-manager-85c4996f8c-z6m6q\" is forbidden: User \"system:node:ip-10-0-141-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-22.ec2.internal' and this object" Jun 02 05:33:39.897211 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:39.897175 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62lnq\" (UniqueName: \"kubernetes.io/projected/8c6f1be9-606c-435a-8dcb-c3fda0677e12-kube-api-access-62lnq\") pod \"8c6f1be9-606c-435a-8dcb-c3fda0677e12\" (UID: \"8c6f1be9-606c-435a-8dcb-c3fda0677e12\") " Jun 02 05:33:39.899496 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:39.899466 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6f1be9-606c-435a-8dcb-c3fda0677e12-kube-api-access-62lnq" (OuterVolumeSpecName: "kube-api-access-62lnq") pod "8c6f1be9-606c-435a-8dcb-c3fda0677e12" (UID: "8c6f1be9-606c-435a-8dcb-c3fda0677e12"). InnerVolumeSpecName "kube-api-access-62lnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:33:39.998125 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:39.998040 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62lnq\" (UniqueName: \"kubernetes.io/projected/8c6f1be9-606c-435a-8dcb-c3fda0677e12-kube-api-access-62lnq\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:33:40.237958 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.237922 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" event={"ID":"ab3ac6ce-4fcf-4b32-b110-baf01995e4b7","Type":"ContainerStarted","Data":"a30c568460e684727234c81403c8cdb3c28c3fe3c8715b8db4db9f258979a1a7"} Jun 02 05:33:40.237958 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.237957 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" event={"ID":"ab3ac6ce-4fcf-4b32-b110-baf01995e4b7","Type":"ContainerStarted","Data":"b8c611a3d554380761e33b3fe31d33eb7bdbe0ad9e15b4ad0908ccf44dd00465"} Jun 02 05:33:40.238307 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.238040 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" Jun 02 05:33:40.239325 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.239305 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" event={"ID":"1057b8a8-6d5d-4775-8b02-53e6e55f1e04","Type":"ContainerStarted","Data":"fbdc8bad5443d5cac2dbac0a26ecdaa538d239124656d792e308b2977cafe673"} Jun 02 05:33:40.240193 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.240164 2561 status_manager.go:895] "Failed to get status for pod" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" err="pods \"limitador-operator-controller-manager-85c4996f8c-z6m6q\" is forbidden: User \"system:node:ip-10-0-141-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-22.ec2.internal' and this object" Jun 02 05:33:40.240416 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.240397 2561 generic.go:358] "Generic (PLEG): container finished" podID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" containerID="ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37" exitCode=0 Jun 02 05:33:40.240493 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.240428 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" Jun 02 05:33:40.240493 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.240447 2561 scope.go:117] "RemoveContainer" containerID="ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37" Jun 02 05:33:40.252087 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.252070 2561 scope.go:117] "RemoveContainer" containerID="ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37" Jun 02 05:33:40.252319 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:33:40.252299 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37\": container with ID starting with ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37 not found: ID does not exist" containerID="ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37" Jun 02 05:33:40.252364 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.252327 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37"} err="failed to get container status \"ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37\": rpc error: code = NotFound desc = could not find container \"ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37\": container with ID starting with ccc682410d738d8a778972432488e2dd4b0ab685ef41aa66b8bb2c7a4c80cd37 not found: ID does not exist" Jun 02 05:33:40.259512 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.259472 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" podStartSLOduration=14.259456098 podStartE2EDuration="14.259456098s" podCreationTimestamp="2026-06-02 05:33:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:33:40.258918904 +0000 UTC m=+471.969368149" watchObservedRunningTime="2026-06-02 05:33:40.259456098 +0000 UTC m=+471.969905342" Jun 02 05:33:40.277801 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.277764 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-55vg7" podStartSLOduration=1.019923895 podStartE2EDuration="25.277756368s" podCreationTimestamp="2026-06-02 05:33:15 +0000 UTC" firstStartedPulling="2026-06-02 05:33:15.498162924 +0000 UTC m=+447.208612148" lastFinishedPulling="2026-06-02 05:33:39.755995387 +0000 UTC m=+471.466444621" observedRunningTime="2026-06-02 05:33:40.275913141 +0000 UTC m=+471.986362387" watchObservedRunningTime="2026-06-02 05:33:40.277756368 +0000 UTC m=+471.988205613" Jun 02 05:33:40.278280 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.278260 2561 status_manager.go:895] "Failed to get status for pod" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-z6m6q" err="pods \"limitador-operator-controller-manager-85c4996f8c-z6m6q\" is forbidden: User \"system:node:ip-10-0-141-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-22.ec2.internal' and this object" Jun 02 05:33:40.786908 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:40.786875 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c6f1be9-606c-435a-8dcb-c3fda0677e12" path="/var/lib/kubelet/pods/8c6f1be9-606c-435a-8dcb-c3fda0677e12/volumes" Jun 02 05:33:49.472939 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.472904 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-555c5c4c65-t7vd7"] Jun 02 05:33:49.531443 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.531412 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-555c5c4c65-t7vd7"] Jun 02 05:33:49.531592 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.531533 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.539629 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.539608 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 05:33:49.539710 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.539636 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 05:33:49.539710 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.539641 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 05:33:49.540364 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.540338 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 05:33:49.540484 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.540469 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 05:33:49.540550 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.540534 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vmd8c\"" Jun 02 05:33:49.545626 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.545611 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 02 05:33:49.677810 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.677774 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-service-ca\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.677990 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.677825 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-oauth-config\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.677990 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.677845 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-trusted-ca-bundle\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.677990 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.677869 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-config\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.677990 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.677901 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-serving-cert\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.677990 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.677915 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fffsh\" (UniqueName: \"kubernetes.io/projected/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-kube-api-access-fffsh\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.677990 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.677980 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-oauth-serving-cert\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779050 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.778952 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-service-ca\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779050 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779015 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-oauth-config\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779050 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779036 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-trusted-ca-bundle\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779056 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-config\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779088 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-serving-cert\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779103 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fffsh\" (UniqueName: \"kubernetes.io/projected/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-kube-api-access-fffsh\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779122 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-oauth-serving-cert\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779821 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779791 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-service-ca\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.779928 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779847 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-oauth-serving-cert\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.780008 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.779993 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-trusted-ca-bundle\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.780076 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.780055 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-config\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.781620 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.781601 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-oauth-config\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.781675 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.781652 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-console-serving-cert\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.789007 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.788953 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fffsh\" (UniqueName: \"kubernetes.io/projected/c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807-kube-api-access-fffsh\") pod \"console-555c5c4c65-t7vd7\" (UID: \"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807\") " pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.840978 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.840924 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:49.974856 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:49.974831 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-555c5c4c65-t7vd7"] Jun 02 05:33:49.977476 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:33:49.977448 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45bf84c_5a08_4bb2_a1ef_8f5f1afc9807.slice/crio-5f570e230da6bcd9d7873c90b3404dec0724cc854b26aa40bc3417ff7f3e0403 WatchSource:0}: Error finding container 5f570e230da6bcd9d7873c90b3404dec0724cc854b26aa40bc3417ff7f3e0403: Status 404 returned error can't find the container with id 5f570e230da6bcd9d7873c90b3404dec0724cc854b26aa40bc3417ff7f3e0403 Jun 02 05:33:50.274176 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:50.274139 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555c5c4c65-t7vd7" event={"ID":"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807","Type":"ContainerStarted","Data":"ccbd827949bd8a129061643b6d7dcd88612d23d8eeec4d8996243a6400b82489"} Jun 02 05:33:50.274176 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:50.274177 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555c5c4c65-t7vd7" event={"ID":"c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807","Type":"ContainerStarted","Data":"5f570e230da6bcd9d7873c90b3404dec0724cc854b26aa40bc3417ff7f3e0403"} Jun 02 05:33:50.296371 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:50.296321 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-555c5c4c65-t7vd7" podStartSLOduration=1.296305603 podStartE2EDuration="1.296305603s" podCreationTimestamp="2026-06-02 05:33:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:33:50.29611778 +0000 UTC m=+482.006567026" watchObservedRunningTime="2026-06-02 05:33:50.296305603 +0000 UTC m=+482.006754852" Jun 02 05:33:51.247365 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:51.247340 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-btkhm" Jun 02 05:33:59.841676 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:59.841643 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:59.842053 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:59.841772 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:33:59.846664 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:33:59.846643 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:34:00.320784 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:00.320755 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-555c5c4c65-t7vd7" Jun 02 05:34:12.270443 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.270407 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-ccqc5"] Jun 02 05:34:12.330808 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.330774 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-ccqc5"] Jun 02 05:34:12.330980 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.330886 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" Jun 02 05:34:12.334872 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.334847 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m87gd\"" Jun 02 05:34:12.354547 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.354521 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5ssg\" (UniqueName: \"kubernetes.io/projected/d98e4e8a-8d37-4594-9acd-d9ab914de8f7-kube-api-access-d5ssg\") pod \"authorino-f99f4b5cd-ccqc5\" (UID: \"d98e4e8a-8d37-4594-9acd-d9ab914de8f7\") " pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" Jun 02 05:34:12.445174 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.445145 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-j7j64"] Jun 02 05:34:12.448976 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.448938 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-j7j64" Jun 02 05:34:12.455113 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.455084 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d5ssg\" (UniqueName: \"kubernetes.io/projected/d98e4e8a-8d37-4594-9acd-d9ab914de8f7-kube-api-access-d5ssg\") pod \"authorino-f99f4b5cd-ccqc5\" (UID: \"d98e4e8a-8d37-4594-9acd-d9ab914de8f7\") " pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" Jun 02 05:34:12.457352 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.457327 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-j7j64"] Jun 02 05:34:12.465797 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.465772 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5ssg\" (UniqueName: \"kubernetes.io/projected/d98e4e8a-8d37-4594-9acd-d9ab914de8f7-kube-api-access-d5ssg\") pod \"authorino-f99f4b5cd-ccqc5\" (UID: \"d98e4e8a-8d37-4594-9acd-d9ab914de8f7\") " pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" Jun 02 05:34:12.556475 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.556449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q77xc\" (UniqueName: \"kubernetes.io/projected/3e49ec38-5e15-4381-9f60-519d11c4d223-kube-api-access-q77xc\") pod \"authorino-7498df8756-j7j64\" (UID: \"3e49ec38-5e15-4381-9f60-519d11c4d223\") " pod="kuadrant-system/authorino-7498df8756-j7j64" Jun 02 05:34:12.640714 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.640686 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" Jun 02 05:34:12.657848 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.657816 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q77xc\" (UniqueName: \"kubernetes.io/projected/3e49ec38-5e15-4381-9f60-519d11c4d223-kube-api-access-q77xc\") pod \"authorino-7498df8756-j7j64\" (UID: \"3e49ec38-5e15-4381-9f60-519d11c4d223\") " pod="kuadrant-system/authorino-7498df8756-j7j64" Jun 02 05:34:12.665995 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.665954 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q77xc\" (UniqueName: \"kubernetes.io/projected/3e49ec38-5e15-4381-9f60-519d11c4d223-kube-api-access-q77xc\") pod \"authorino-7498df8756-j7j64\" (UID: \"3e49ec38-5e15-4381-9f60-519d11c4d223\") " pod="kuadrant-system/authorino-7498df8756-j7j64" Jun 02 05:34:12.759581 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.759552 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-j7j64" Jun 02 05:34:12.765921 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.765901 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-ccqc5"] Jun 02 05:34:12.768317 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:34:12.768288 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd98e4e8a_8d37_4594_9acd_d9ab914de8f7.slice/crio-ba07da471dd40e0e103513e7e09a52738ace5ee662b29ac71ab8c3a4b44b83d4 WatchSource:0}: Error finding container ba07da471dd40e0e103513e7e09a52738ace5ee662b29ac71ab8c3a4b44b83d4: Status 404 returned error can't find the container with id ba07da471dd40e0e103513e7e09a52738ace5ee662b29ac71ab8c3a4b44b83d4 Jun 02 05:34:12.885053 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:12.885024 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-j7j64"] Jun 02 05:34:12.887848 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:34:12.887827 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e49ec38_5e15_4381_9f60_519d11c4d223.slice/crio-fe0a72cb9d1214f2e82ceb473bb29dd0574cb78776a94b617cd3d08c7ceb29ed WatchSource:0}: Error finding container fe0a72cb9d1214f2e82ceb473bb29dd0574cb78776a94b617cd3d08c7ceb29ed: Status 404 returned error can't find the container with id fe0a72cb9d1214f2e82ceb473bb29dd0574cb78776a94b617cd3d08c7ceb29ed Jun 02 05:34:13.354229 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:13.354195 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" event={"ID":"d98e4e8a-8d37-4594-9acd-d9ab914de8f7","Type":"ContainerStarted","Data":"ba07da471dd40e0e103513e7e09a52738ace5ee662b29ac71ab8c3a4b44b83d4"} Jun 02 05:34:13.355125 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:13.355105 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-j7j64" event={"ID":"3e49ec38-5e15-4381-9f60-519d11c4d223","Type":"ContainerStarted","Data":"fe0a72cb9d1214f2e82ceb473bb29dd0574cb78776a94b617cd3d08c7ceb29ed"} Jun 02 05:34:16.370375 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:16.370332 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" event={"ID":"d98e4e8a-8d37-4594-9acd-d9ab914de8f7","Type":"ContainerStarted","Data":"898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa"} Jun 02 05:34:16.371950 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:16.371905 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-j7j64" event={"ID":"3e49ec38-5e15-4381-9f60-519d11c4d223","Type":"ContainerStarted","Data":"8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6"} Jun 02 05:34:16.388372 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:16.388154 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" podStartSLOduration=0.858884285 podStartE2EDuration="4.388136132s" podCreationTimestamp="2026-06-02 05:34:12 +0000 UTC" firstStartedPulling="2026-06-02 05:34:12.769650942 +0000 UTC m=+504.480100167" lastFinishedPulling="2026-06-02 05:34:16.298902783 +0000 UTC m=+508.009352014" observedRunningTime="2026-06-02 05:34:16.386403983 +0000 UTC m=+508.096853230" watchObservedRunningTime="2026-06-02 05:34:16.388136132 +0000 UTC m=+508.098585382" Jun 02 05:34:16.403934 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:16.402630 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-j7j64" podStartSLOduration=1.003589056 podStartE2EDuration="4.40261092s" podCreationTimestamp="2026-06-02 05:34:12 +0000 UTC" firstStartedPulling="2026-06-02 05:34:12.889128832 +0000 UTC m=+504.599578056" lastFinishedPulling="2026-06-02 05:34:16.288150695 +0000 UTC m=+507.998599920" observedRunningTime="2026-06-02 05:34:16.401416809 +0000 UTC m=+508.111866069" watchObservedRunningTime="2026-06-02 05:34:16.40261092 +0000 UTC m=+508.113060167" Jun 02 05:34:16.438809 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:16.438772 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-ccqc5"] Jun 02 05:34:18.378295 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:18.378247 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" podUID="d98e4e8a-8d37-4594-9acd-d9ab914de8f7" containerName="authorino" containerID="cri-o://898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa" gracePeriod=30 Jun 02 05:34:18.617870 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:18.617846 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" Jun 02 05:34:18.711486 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:18.711411 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5ssg\" (UniqueName: \"kubernetes.io/projected/d98e4e8a-8d37-4594-9acd-d9ab914de8f7-kube-api-access-d5ssg\") pod \"d98e4e8a-8d37-4594-9acd-d9ab914de8f7\" (UID: \"d98e4e8a-8d37-4594-9acd-d9ab914de8f7\") " Jun 02 05:34:18.713556 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:18.713530 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98e4e8a-8d37-4594-9acd-d9ab914de8f7-kube-api-access-d5ssg" (OuterVolumeSpecName: "kube-api-access-d5ssg") pod "d98e4e8a-8d37-4594-9acd-d9ab914de8f7" (UID: "d98e4e8a-8d37-4594-9acd-d9ab914de8f7"). InnerVolumeSpecName "kube-api-access-d5ssg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:34:18.812051 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:18.812016 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5ssg\" (UniqueName: \"kubernetes.io/projected/d98e4e8a-8d37-4594-9acd-d9ab914de8f7-kube-api-access-d5ssg\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:34:19.382543 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.382506 2561 generic.go:358] "Generic (PLEG): container finished" podID="d98e4e8a-8d37-4594-9acd-d9ab914de8f7" containerID="898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa" exitCode=0 Jun 02 05:34:19.382977 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.382551 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" event={"ID":"d98e4e8a-8d37-4594-9acd-d9ab914de8f7","Type":"ContainerDied","Data":"898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa"} Jun 02 05:34:19.382977 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.382557 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" Jun 02 05:34:19.382977 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.382574 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-ccqc5" event={"ID":"d98e4e8a-8d37-4594-9acd-d9ab914de8f7","Type":"ContainerDied","Data":"ba07da471dd40e0e103513e7e09a52738ace5ee662b29ac71ab8c3a4b44b83d4"} Jun 02 05:34:19.382977 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.382588 2561 scope.go:117] "RemoveContainer" containerID="898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa" Jun 02 05:34:19.390785 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.390767 2561 scope.go:117] "RemoveContainer" containerID="898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa" Jun 02 05:34:19.391059 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:34:19.391041 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa\": container with ID starting with 898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa not found: ID does not exist" containerID="898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa" Jun 02 05:34:19.391115 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.391072 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa"} err="failed to get container status \"898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa\": rpc error: code = NotFound desc = could not find container \"898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa\": container with ID starting with 898f90af4fc62b88982137b8de87b099d968f1a0bffabd9f1a0835bcb141afaa not found: ID does not exist" Jun 02 05:34:19.402142 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.402119 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-ccqc5"] Jun 02 05:34:19.405477 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:19.405456 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-ccqc5"] Jun 02 05:34:20.786714 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:34:20.786681 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d98e4e8a-8d37-4594-9acd-d9ab914de8f7" path="/var/lib/kubelet/pods/d98e4e8a-8d37-4594-9acd-d9ab914de8f7/volumes" Jun 02 05:35:02.990683 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.990646 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:35:02.991176 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.991021 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d98e4e8a-8d37-4594-9acd-d9ab914de8f7" containerName="authorino" Jun 02 05:35:02.991176 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.991035 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d98e4e8a-8d37-4594-9acd-d9ab914de8f7" containerName="authorino" Jun 02 05:35:02.991176 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.991094 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="d98e4e8a-8d37-4594-9acd-d9ab914de8f7" containerName="authorino" Jun 02 05:35:02.995977 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.995944 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:02.998485 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.998460 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 05:35:02.998485 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.998467 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 05:35:02.999473 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.999451 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-xvbfm\"" Jun 02 05:35:02.999558 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:02.999519 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 05:35:03.006271 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:03.006248 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:35:03.183297 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:03.183262 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ld5s\" (UniqueName: \"kubernetes.io/projected/a96059bd-4e1f-40b8-b6de-8e8b657fc43a-kube-api-access-8ld5s\") pod \"maas-keycloak-0\" (UID: \"a96059bd-4e1f-40b8-b6de-8e8b657fc43a\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:03.284587 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:03.284503 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8ld5s\" (UniqueName: \"kubernetes.io/projected/a96059bd-4e1f-40b8-b6de-8e8b657fc43a-kube-api-access-8ld5s\") pod \"maas-keycloak-0\" (UID: \"a96059bd-4e1f-40b8-b6de-8e8b657fc43a\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:03.293751 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:03.293721 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ld5s\" (UniqueName: \"kubernetes.io/projected/a96059bd-4e1f-40b8-b6de-8e8b657fc43a-kube-api-access-8ld5s\") pod \"maas-keycloak-0\" (UID: \"a96059bd-4e1f-40b8-b6de-8e8b657fc43a\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:03.306533 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:03.306496 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:03.436764 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:03.436734 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:35:03.439376 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:35:03.439345 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda96059bd_4e1f_40b8_b6de_8e8b657fc43a.slice/crio-aadb02dcfff94823a82053297d2e92b52631d00d0daa204f5b2af424d4bce96a WatchSource:0}: Error finding container aadb02dcfff94823a82053297d2e92b52631d00d0daa204f5b2af424d4bce96a: Status 404 returned error can't find the container with id aadb02dcfff94823a82053297d2e92b52631d00d0daa204f5b2af424d4bce96a Jun 02 05:35:03.520984 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:03.520936 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"a96059bd-4e1f-40b8-b6de-8e8b657fc43a","Type":"ContainerStarted","Data":"aadb02dcfff94823a82053297d2e92b52631d00d0daa204f5b2af424d4bce96a"} Jun 02 05:35:09.544140 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:09.544100 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"a96059bd-4e1f-40b8-b6de-8e8b657fc43a","Type":"ContainerStarted","Data":"90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4"} Jun 02 05:35:09.565096 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:09.565045 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.28740774 podStartE2EDuration="7.565029864s" podCreationTimestamp="2026-06-02 05:35:02 +0000 UTC" firstStartedPulling="2026-06-02 05:35:03.440632974 +0000 UTC m=+555.151082199" lastFinishedPulling="2026-06-02 05:35:08.7182551 +0000 UTC m=+560.428704323" observedRunningTime="2026-06-02 05:35:09.562693313 +0000 UTC m=+561.273142559" watchObservedRunningTime="2026-06-02 05:35:09.565029864 +0000 UTC m=+561.275479109" Jun 02 05:35:10.307531 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:10.307486 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:10.309457 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:10.309399 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:11.307059 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:11.307004 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:12.307676 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:12.307627 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:13.307023 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:13.306955 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:13.307354 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:13.307318 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:14.307897 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:14.307850 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:15.307391 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:15.307335 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:16.307200 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:16.307145 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:17.307233 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:17.307183 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:18.307870 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:18.307811 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:19.307936 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:19.307879 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:20.307462 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:20.307421 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 02 05:35:21.412025 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:21.411982 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:21.427191 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:21.427143 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 05:35:31.418256 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:31.418211 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 05:35:32.385765 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.385730 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-fgjlp"] Jun 02 05:35:32.397240 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.397214 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-fgjlp"] Jun 02 05:35:32.397376 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.397330 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-fgjlp" Jun 02 05:35:32.549754 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.549721 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvfw9\" (UniqueName: \"kubernetes.io/projected/cd44d850-b215-4d3f-af15-a6ef84b68532-kube-api-access-nvfw9\") pod \"authorino-8b475cf9f-fgjlp\" (UID: \"cd44d850-b215-4d3f-af15-a6ef84b68532\") " pod="kuadrant-system/authorino-8b475cf9f-fgjlp" Jun 02 05:35:32.615473 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.615432 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-fgjlp"] Jun 02 05:35:32.615698 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:35:32.615673 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nvfw9], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8b475cf9f-fgjlp" podUID="cd44d850-b215-4d3f-af15-a6ef84b68532" Jun 02 05:35:32.629385 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.629364 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-fgjlp" Jun 02 05:35:32.633614 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.633596 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-fgjlp" Jun 02 05:35:32.650411 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.650351 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b69cfd6c4-d4dv6"] Jun 02 05:35:32.650652 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.650627 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nvfw9\" (UniqueName: \"kubernetes.io/projected/cd44d850-b215-4d3f-af15-a6ef84b68532-kube-api-access-nvfw9\") pod \"authorino-8b475cf9f-fgjlp\" (UID: \"cd44d850-b215-4d3f-af15-a6ef84b68532\") " pod="kuadrant-system/authorino-8b475cf9f-fgjlp" Jun 02 05:35:32.653564 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.653548 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b69cfd6c4-d4dv6" Jun 02 05:35:32.664033 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.664012 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b69cfd6c4-d4dv6"] Jun 02 05:35:32.666748 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.666726 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvfw9\" (UniqueName: \"kubernetes.io/projected/cd44d850-b215-4d3f-af15-a6ef84b68532-kube-api-access-nvfw9\") pod \"authorino-8b475cf9f-fgjlp\" (UID: \"cd44d850-b215-4d3f-af15-a6ef84b68532\") " pod="kuadrant-system/authorino-8b475cf9f-fgjlp" Jun 02 05:35:32.727069 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.727040 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b69cfd6c4-d4dv6"] Jun 02 05:35:32.727269 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:35:32.727250 2561 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-z2kks], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6b69cfd6c4-d4dv6" podUID="5b384ebc-7fea-4173-92a6-bf919a0a4cea" Jun 02 05:35:32.751251 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.751221 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvfw9\" (UniqueName: \"kubernetes.io/projected/cd44d850-b215-4d3f-af15-a6ef84b68532-kube-api-access-nvfw9\") pod \"cd44d850-b215-4d3f-af15-a6ef84b68532\" (UID: \"cd44d850-b215-4d3f-af15-a6ef84b68532\") " Jun 02 05:35:32.751376 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.751362 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2kks\" (UniqueName: \"kubernetes.io/projected/5b384ebc-7fea-4173-92a6-bf919a0a4cea-kube-api-access-z2kks\") pod \"authorino-6b69cfd6c4-d4dv6\" (UID: \"5b384ebc-7fea-4173-92a6-bf919a0a4cea\") " pod="kuadrant-system/authorino-6b69cfd6c4-d4dv6" Jun 02 05:35:32.753406 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.753380 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd44d850-b215-4d3f-af15-a6ef84b68532-kube-api-access-nvfw9" (OuterVolumeSpecName: "kube-api-access-nvfw9") pod "cd44d850-b215-4d3f-af15-a6ef84b68532" (UID: "cd44d850-b215-4d3f-af15-a6ef84b68532"). InnerVolumeSpecName "kube-api-access-nvfw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:35:32.756041 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.756021 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c94cb989d-zsxmg"] Jun 02 05:35:32.759418 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.759401 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:35:32.762446 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.762417 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 02 05:35:32.770755 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.770735 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c94cb989d-zsxmg"] Jun 02 05:35:32.852518 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.852483 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c26742c-bba8-4f7c-98f4-06a0082346ee-tls-cert\") pod \"authorino-c94cb989d-zsxmg\" (UID: \"1c26742c-bba8-4f7c-98f4-06a0082346ee\") " pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:35:32.852676 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.852600 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z2kks\" (UniqueName: \"kubernetes.io/projected/5b384ebc-7fea-4173-92a6-bf919a0a4cea-kube-api-access-z2kks\") pod \"authorino-6b69cfd6c4-d4dv6\" (UID: \"5b384ebc-7fea-4173-92a6-bf919a0a4cea\") " pod="kuadrant-system/authorino-6b69cfd6c4-d4dv6" Jun 02 05:35:32.852676 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.852651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm67k\" (UniqueName: \"kubernetes.io/projected/1c26742c-bba8-4f7c-98f4-06a0082346ee-kube-api-access-vm67k\") pod \"authorino-c94cb989d-zsxmg\" (UID: \"1c26742c-bba8-4f7c-98f4-06a0082346ee\") " pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:35:32.852792 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.852702 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvfw9\" (UniqueName: \"kubernetes.io/projected/cd44d850-b215-4d3f-af15-a6ef84b68532-kube-api-access-nvfw9\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:35:32.862022 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.862001 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2kks\" (UniqueName: \"kubernetes.io/projected/5b384ebc-7fea-4173-92a6-bf919a0a4cea-kube-api-access-z2kks\") pod \"authorino-6b69cfd6c4-d4dv6\" (UID: \"5b384ebc-7fea-4173-92a6-bf919a0a4cea\") " pod="kuadrant-system/authorino-6b69cfd6c4-d4dv6" Jun 02 05:35:32.953183 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.953095 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vm67k\" (UniqueName: \"kubernetes.io/projected/1c26742c-bba8-4f7c-98f4-06a0082346ee-kube-api-access-vm67k\") pod \"authorino-c94cb989d-zsxmg\" (UID: \"1c26742c-bba8-4f7c-98f4-06a0082346ee\") " pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:35:32.953183 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.953149 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c26742c-bba8-4f7c-98f4-06a0082346ee-tls-cert\") pod \"authorino-c94cb989d-zsxmg\" (UID: \"1c26742c-bba8-4f7c-98f4-06a0082346ee\") " pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:35:32.955654 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.955634 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c26742c-bba8-4f7c-98f4-06a0082346ee-tls-cert\") pod \"authorino-c94cb989d-zsxmg\" (UID: \"1c26742c-bba8-4f7c-98f4-06a0082346ee\") " pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:35:32.964172 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:32.964149 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm67k\" (UniqueName: \"kubernetes.io/projected/1c26742c-bba8-4f7c-98f4-06a0082346ee-kube-api-access-vm67k\") pod \"authorino-c94cb989d-zsxmg\" (UID: \"1c26742c-bba8-4f7c-98f4-06a0082346ee\") " pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:35:33.068144 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.068110 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:35:33.193001 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.192952 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c94cb989d-zsxmg"] Jun 02 05:35:33.196457 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:35:33.196427 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c26742c_bba8_4f7c_98f4_06a0082346ee.slice/crio-481e41385f2f537d24bf056b8081abea3c46f285f35abff82d1814996f1af04b WatchSource:0}: Error finding container 481e41385f2f537d24bf056b8081abea3c46f285f35abff82d1814996f1af04b: Status 404 returned error can't find the container with id 481e41385f2f537d24bf056b8081abea3c46f285f35abff82d1814996f1af04b Jun 02 05:35:33.634501 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.634467 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c94cb989d-zsxmg" event={"ID":"1c26742c-bba8-4f7c-98f4-06a0082346ee","Type":"ContainerStarted","Data":"0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d"} Jun 02 05:35:33.634907 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.634508 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c94cb989d-zsxmg" event={"ID":"1c26742c-bba8-4f7c-98f4-06a0082346ee","Type":"ContainerStarted","Data":"481e41385f2f537d24bf056b8081abea3c46f285f35abff82d1814996f1af04b"} Jun 02 05:35:33.634907 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.634547 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b69cfd6c4-d4dv6" Jun 02 05:35:33.634907 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.634594 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-fgjlp" Jun 02 05:35:33.640523 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.640503 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b69cfd6c4-d4dv6" Jun 02 05:35:33.650873 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.650808 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-c94cb989d-zsxmg" podStartSLOduration=1.298282183 podStartE2EDuration="1.650791273s" podCreationTimestamp="2026-06-02 05:35:32 +0000 UTC" firstStartedPulling="2026-06-02 05:35:33.197635113 +0000 UTC m=+584.908084336" lastFinishedPulling="2026-06-02 05:35:33.550144203 +0000 UTC m=+585.260593426" observedRunningTime="2026-06-02 05:35:33.649605278 +0000 UTC m=+585.360054525" watchObservedRunningTime="2026-06-02 05:35:33.650791273 +0000 UTC m=+585.361240520" Jun 02 05:35:33.676064 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.676011 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-fgjlp"] Jun 02 05:35:33.680736 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.680710 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-j7j64"] Jun 02 05:35:33.680928 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.680900 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-j7j64" podUID="3e49ec38-5e15-4381-9f60-519d11c4d223" containerName="authorino" containerID="cri-o://8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6" gracePeriod=30 Jun 02 05:35:33.682919 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.682894 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-fgjlp"] Jun 02 05:35:33.759012 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.758950 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2kks\" (UniqueName: \"kubernetes.io/projected/5b384ebc-7fea-4173-92a6-bf919a0a4cea-kube-api-access-z2kks\") pod \"5b384ebc-7fea-4173-92a6-bf919a0a4cea\" (UID: \"5b384ebc-7fea-4173-92a6-bf919a0a4cea\") " Jun 02 05:35:33.761140 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.761118 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b384ebc-7fea-4173-92a6-bf919a0a4cea-kube-api-access-z2kks" (OuterVolumeSpecName: "kube-api-access-z2kks") pod "5b384ebc-7fea-4173-92a6-bf919a0a4cea" (UID: "5b384ebc-7fea-4173-92a6-bf919a0a4cea"). InnerVolumeSpecName "kube-api-access-z2kks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:35:33.860517 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.860484 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2kks\" (UniqueName: \"kubernetes.io/projected/5b384ebc-7fea-4173-92a6-bf919a0a4cea-kube-api-access-z2kks\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:35:33.933441 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:33.933416 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-j7j64" Jun 02 05:35:34.061923 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.061885 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q77xc\" (UniqueName: \"kubernetes.io/projected/3e49ec38-5e15-4381-9f60-519d11c4d223-kube-api-access-q77xc\") pod \"3e49ec38-5e15-4381-9f60-519d11c4d223\" (UID: \"3e49ec38-5e15-4381-9f60-519d11c4d223\") " Jun 02 05:35:34.064147 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.064121 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e49ec38-5e15-4381-9f60-519d11c4d223-kube-api-access-q77xc" (OuterVolumeSpecName: "kube-api-access-q77xc") pod "3e49ec38-5e15-4381-9f60-519d11c4d223" (UID: "3e49ec38-5e15-4381-9f60-519d11c4d223"). InnerVolumeSpecName "kube-api-access-q77xc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:35:34.163534 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.163456 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q77xc\" (UniqueName: \"kubernetes.io/projected/3e49ec38-5e15-4381-9f60-519d11c4d223-kube-api-access-q77xc\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:35:34.638530 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.638492 2561 generic.go:358] "Generic (PLEG): container finished" podID="3e49ec38-5e15-4381-9f60-519d11c4d223" containerID="8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6" exitCode=0 Jun 02 05:35:34.638937 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.638540 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-j7j64" Jun 02 05:35:34.638937 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.638574 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-j7j64" event={"ID":"3e49ec38-5e15-4381-9f60-519d11c4d223","Type":"ContainerDied","Data":"8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6"} Jun 02 05:35:34.638937 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.638612 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-j7j64" event={"ID":"3e49ec38-5e15-4381-9f60-519d11c4d223","Type":"ContainerDied","Data":"fe0a72cb9d1214f2e82ceb473bb29dd0574cb78776a94b617cd3d08c7ceb29ed"} Jun 02 05:35:34.638937 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.638628 2561 scope.go:117] "RemoveContainer" containerID="8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6" Jun 02 05:35:34.638937 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.638773 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b69cfd6c4-d4dv6" Jun 02 05:35:34.651290 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.651245 2561 scope.go:117] "RemoveContainer" containerID="8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6" Jun 02 05:35:34.651607 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:35:34.651583 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6\": container with ID starting with 8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6 not found: ID does not exist" containerID="8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6" Jun 02 05:35:34.651681 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.651618 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6"} err="failed to get container status \"8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6\": rpc error: code = NotFound desc = could not find container \"8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6\": container with ID starting with 8d02e8933a28f58177a181427d1893dee853fc43d8f5362f4cf737aadc05eac6 not found: ID does not exist" Jun 02 05:35:34.676135 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.676109 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b69cfd6c4-d4dv6"] Jun 02 05:35:34.678329 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.678312 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6b69cfd6c4-d4dv6"] Jun 02 05:35:34.694062 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.694031 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-j7j64"] Jun 02 05:35:34.696790 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.696768 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-j7j64"] Jun 02 05:35:34.786242 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.786202 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e49ec38-5e15-4381-9f60-519d11c4d223" path="/var/lib/kubelet/pods/3e49ec38-5e15-4381-9f60-519d11c4d223/volumes" Jun 02 05:35:34.786584 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.786565 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b384ebc-7fea-4173-92a6-bf919a0a4cea" path="/var/lib/kubelet/pods/5b384ebc-7fea-4173-92a6-bf919a0a4cea/volumes" Jun 02 05:35:34.786808 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:34.786792 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd44d850-b215-4d3f-af15-a6ef84b68532" path="/var/lib/kubelet/pods/cd44d850-b215-4d3f-af15-a6ef84b68532/volumes" Jun 02 05:35:36.851889 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.851847 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5cbf5b65f-ppxwl"] Jun 02 05:35:36.852270 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.852176 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e49ec38-5e15-4381-9f60-519d11c4d223" containerName="authorino" Jun 02 05:35:36.852270 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.852188 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e49ec38-5e15-4381-9f60-519d11c4d223" containerName="authorino" Jun 02 05:35:36.852270 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.852248 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="3e49ec38-5e15-4381-9f60-519d11c4d223" containerName="authorino" Jun 02 05:35:36.855070 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.855045 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:36.858906 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.858884 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m2qd5\"" Jun 02 05:35:36.859087 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.859069 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 05:35:36.866622 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.866592 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cbf5b65f-ppxwl"] Jun 02 05:35:36.988935 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:36.988905 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk76m\" (UniqueName: \"kubernetes.io/projected/347a243b-f08d-43ff-8275-2286293f96fc-kube-api-access-xk76m\") pod \"maas-controller-5cbf5b65f-ppxwl\" (UID: \"347a243b-f08d-43ff-8275-2286293f96fc\") " pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:37.009317 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.009290 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-5fb4m"] Jun 02 05:35:37.012629 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.012615 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:35:37.021563 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.021539 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-5fb4m"] Jun 02 05:35:37.089563 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.089528 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xk76m\" (UniqueName: \"kubernetes.io/projected/347a243b-f08d-43ff-8275-2286293f96fc-kube-api-access-xk76m\") pod \"maas-controller-5cbf5b65f-ppxwl\" (UID: \"347a243b-f08d-43ff-8275-2286293f96fc\") " pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:37.097951 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.097921 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk76m\" (UniqueName: \"kubernetes.io/projected/347a243b-f08d-43ff-8275-2286293f96fc-kube-api-access-xk76m\") pod \"maas-controller-5cbf5b65f-ppxwl\" (UID: \"347a243b-f08d-43ff-8275-2286293f96fc\") " pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:37.171330 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.171230 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:37.190392 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.190347 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6qwt\" (UniqueName: \"kubernetes.io/projected/5c998e38-4090-4948-b86e-c51e00aeb74d-kube-api-access-l6qwt\") pod \"maas-controller-7bc6dfc4b6-5fb4m\" (UID: \"5c998e38-4090-4948-b86e-c51e00aeb74d\") " pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:35:37.291241 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.291211 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l6qwt\" (UniqueName: \"kubernetes.io/projected/5c998e38-4090-4948-b86e-c51e00aeb74d-kube-api-access-l6qwt\") pod \"maas-controller-7bc6dfc4b6-5fb4m\" (UID: \"5c998e38-4090-4948-b86e-c51e00aeb74d\") " pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:35:37.300155 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.300129 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5cbf5b65f-ppxwl"] Jun 02 05:35:37.302938 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:35:37.302912 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347a243b_f08d_43ff_8275_2286293f96fc.slice/crio-142fb986fd2f68953a0593e2c81ed70ad418c6b6e0f6f42d549532bc879f1375 WatchSource:0}: Error finding container 142fb986fd2f68953a0593e2c81ed70ad418c6b6e0f6f42d549532bc879f1375: Status 404 returned error can't find the container with id 142fb986fd2f68953a0593e2c81ed70ad418c6b6e0f6f42d549532bc879f1375 Jun 02 05:35:37.303098 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.303079 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6qwt\" (UniqueName: \"kubernetes.io/projected/5c998e38-4090-4948-b86e-c51e00aeb74d-kube-api-access-l6qwt\") pod \"maas-controller-7bc6dfc4b6-5fb4m\" (UID: \"5c998e38-4090-4948-b86e-c51e00aeb74d\") " pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:35:37.323038 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.323013 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:35:37.448491 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.448404 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-5fb4m"] Jun 02 05:35:37.451025 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:35:37.450998 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c998e38_4090_4948_b86e_c51e00aeb74d.slice/crio-f57790f1ec24a8ba3fbfc1eab16f63616117eb18d285f40d6ad6325459d870f7 WatchSource:0}: Error finding container f57790f1ec24a8ba3fbfc1eab16f63616117eb18d285f40d6ad6325459d870f7: Status 404 returned error can't find the container with id f57790f1ec24a8ba3fbfc1eab16f63616117eb18d285f40d6ad6325459d870f7 Jun 02 05:35:37.649958 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.649922 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" event={"ID":"5c998e38-4090-4948-b86e-c51e00aeb74d","Type":"ContainerStarted","Data":"f57790f1ec24a8ba3fbfc1eab16f63616117eb18d285f40d6ad6325459d870f7"} Jun 02 05:35:37.650861 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:37.650836 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" event={"ID":"347a243b-f08d-43ff-8275-2286293f96fc","Type":"ContainerStarted","Data":"142fb986fd2f68953a0593e2c81ed70ad418c6b6e0f6f42d549532bc879f1375"} Jun 02 05:35:40.664721 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:40.664678 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" event={"ID":"5c998e38-4090-4948-b86e-c51e00aeb74d","Type":"ContainerStarted","Data":"1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030"} Jun 02 05:35:40.665179 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:40.664817 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:35:40.666040 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:40.666013 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" event={"ID":"347a243b-f08d-43ff-8275-2286293f96fc","Type":"ContainerStarted","Data":"4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa"} Jun 02 05:35:40.666151 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:40.666137 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:40.683514 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:40.683472 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" podStartSLOduration=2.144667243 podStartE2EDuration="4.683458944s" podCreationTimestamp="2026-06-02 05:35:36 +0000 UTC" firstStartedPulling="2026-06-02 05:35:37.452284763 +0000 UTC m=+589.162733987" lastFinishedPulling="2026-06-02 05:35:39.991076464 +0000 UTC m=+591.701525688" observedRunningTime="2026-06-02 05:35:40.68275298 +0000 UTC m=+592.393202228" watchObservedRunningTime="2026-06-02 05:35:40.683458944 +0000 UTC m=+592.393908193" Jun 02 05:35:40.704343 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:40.704302 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" podStartSLOduration=2.036968629 podStartE2EDuration="4.704290536s" podCreationTimestamp="2026-06-02 05:35:36 +0000 UTC" firstStartedPulling="2026-06-02 05:35:37.304578461 +0000 UTC m=+589.015027685" lastFinishedPulling="2026-06-02 05:35:39.971900366 +0000 UTC m=+591.682349592" observedRunningTime="2026-06-02 05:35:40.703918163 +0000 UTC m=+592.414367408" watchObservedRunningTime="2026-06-02 05:35:40.704290536 +0000 UTC m=+592.414739782" Jun 02 05:35:51.675114 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:51.675082 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:35:51.675505 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:51.675131 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:51.735761 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:51.735724 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cbf5b65f-ppxwl"] Jun 02 05:35:51.736009 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:51.735949 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" podUID="347a243b-f08d-43ff-8275-2286293f96fc" containerName="manager" containerID="cri-o://4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa" gracePeriod=30 Jun 02 05:35:51.976381 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:51.976357 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:52.011424 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.011393 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk76m\" (UniqueName: \"kubernetes.io/projected/347a243b-f08d-43ff-8275-2286293f96fc-kube-api-access-xk76m\") pod \"347a243b-f08d-43ff-8275-2286293f96fc\" (UID: \"347a243b-f08d-43ff-8275-2286293f96fc\") " Jun 02 05:35:52.013570 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.013543 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347a243b-f08d-43ff-8275-2286293f96fc-kube-api-access-xk76m" (OuterVolumeSpecName: "kube-api-access-xk76m") pod "347a243b-f08d-43ff-8275-2286293f96fc" (UID: "347a243b-f08d-43ff-8275-2286293f96fc"). InnerVolumeSpecName "kube-api-access-xk76m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:35:52.112567 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.112526 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk76m\" (UniqueName: \"kubernetes.io/projected/347a243b-f08d-43ff-8275-2286293f96fc-kube-api-access-xk76m\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:35:52.708369 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.708331 2561 generic.go:358] "Generic (PLEG): container finished" podID="347a243b-f08d-43ff-8275-2286293f96fc" containerID="4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa" exitCode=0 Jun 02 05:35:52.708793 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.708391 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" Jun 02 05:35:52.708793 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.708422 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" event={"ID":"347a243b-f08d-43ff-8275-2286293f96fc","Type":"ContainerDied","Data":"4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa"} Jun 02 05:35:52.708793 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.708462 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5cbf5b65f-ppxwl" event={"ID":"347a243b-f08d-43ff-8275-2286293f96fc","Type":"ContainerDied","Data":"142fb986fd2f68953a0593e2c81ed70ad418c6b6e0f6f42d549532bc879f1375"} Jun 02 05:35:52.708793 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.708479 2561 scope.go:117] "RemoveContainer" containerID="4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa" Jun 02 05:35:52.716611 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.716597 2561 scope.go:117] "RemoveContainer" containerID="4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa" Jun 02 05:35:52.716877 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:35:52.716860 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa\": container with ID starting with 4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa not found: ID does not exist" containerID="4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa" Jun 02 05:35:52.716940 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.716887 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa"} err="failed to get container status \"4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa\": rpc error: code = NotFound desc = could not find container \"4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa\": container with ID starting with 4302452610a8315db96da11de7098857f782ff37b33fdd98209d717e78920eaa not found: ID does not exist" Jun 02 05:35:52.729077 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.729054 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5cbf5b65f-ppxwl"] Jun 02 05:35:52.734024 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.733993 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5cbf5b65f-ppxwl"] Jun 02 05:35:52.786807 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:35:52.786781 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="347a243b-f08d-43ff-8275-2286293f96fc" path="/var/lib/kubelet/pods/347a243b-f08d-43ff-8275-2286293f96fc/volumes" Jun 02 05:36:13.052382 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.052344 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-649d57b67d-4tp6b"] Jun 02 05:36:13.055528 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.052844 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347a243b-f08d-43ff-8275-2286293f96fc" containerName="manager" Jun 02 05:36:13.055528 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.052860 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="347a243b-f08d-43ff-8275-2286293f96fc" containerName="manager" Jun 02 05:36:13.055528 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.052948 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="347a243b-f08d-43ff-8275-2286293f96fc" containerName="manager" Jun 02 05:36:13.056827 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.056810 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:13.059565 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.059540 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 02 05:36:13.073087 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.066594 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-649d57b67d-4tp6b"] Jun 02 05:36:13.159924 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.159890 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-maas-api-tls\") pod \"maas-api-649d57b67d-4tp6b\" (UID: \"bab5d774-5c5d-4bbc-85a9-a2c16961df4b\") " pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:13.159924 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.159927 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cnss\" (UniqueName: \"kubernetes.io/projected/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-kube-api-access-4cnss\") pod \"maas-api-649d57b67d-4tp6b\" (UID: \"bab5d774-5c5d-4bbc-85a9-a2c16961df4b\") " pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:13.260789 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.260755 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-maas-api-tls\") pod \"maas-api-649d57b67d-4tp6b\" (UID: \"bab5d774-5c5d-4bbc-85a9-a2c16961df4b\") " pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:13.260789 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.260793 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4cnss\" (UniqueName: \"kubernetes.io/projected/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-kube-api-access-4cnss\") pod \"maas-api-649d57b67d-4tp6b\" (UID: \"bab5d774-5c5d-4bbc-85a9-a2c16961df4b\") " pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:13.261016 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:36:13.260901 2561 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 02 05:36:13.261016 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:36:13.260988 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-maas-api-tls podName:bab5d774-5c5d-4bbc-85a9-a2c16961df4b nodeName:}" failed. No retries permitted until 2026-06-02 05:36:13.760954724 +0000 UTC m=+625.471403954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-maas-api-tls") pod "maas-api-649d57b67d-4tp6b" (UID: "bab5d774-5c5d-4bbc-85a9-a2c16961df4b") : secret "maas-api-serving-cert" not found Jun 02 05:36:13.271216 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.271189 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cnss\" (UniqueName: \"kubernetes.io/projected/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-kube-api-access-4cnss\") pod \"maas-api-649d57b67d-4tp6b\" (UID: \"bab5d774-5c5d-4bbc-85a9-a2c16961df4b\") " pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:13.765189 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.765149 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-maas-api-tls\") pod \"maas-api-649d57b67d-4tp6b\" (UID: \"bab5d774-5c5d-4bbc-85a9-a2c16961df4b\") " pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:13.767720 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.767691 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab5d774-5c5d-4bbc-85a9-a2c16961df4b-maas-api-tls\") pod \"maas-api-649d57b67d-4tp6b\" (UID: \"bab5d774-5c5d-4bbc-85a9-a2c16961df4b\") " pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:13.968361 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:13.968326 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:14.099563 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:14.099534 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-649d57b67d-4tp6b"] Jun 02 05:36:14.102074 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:36:14.102032 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab5d774_5c5d_4bbc_85a9_a2c16961df4b.slice/crio-488dc34f944f276e64851a3d6db5cd788ca9ac247a12ea4bc3b3fd81d2bae75b WatchSource:0}: Error finding container 488dc34f944f276e64851a3d6db5cd788ca9ac247a12ea4bc3b3fd81d2bae75b: Status 404 returned error can't find the container with id 488dc34f944f276e64851a3d6db5cd788ca9ac247a12ea4bc3b3fd81d2bae75b Jun 02 05:36:14.103197 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:14.103179 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 05:36:14.787247 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:14.787208 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-649d57b67d-4tp6b" event={"ID":"bab5d774-5c5d-4bbc-85a9-a2c16961df4b","Type":"ContainerStarted","Data":"488dc34f944f276e64851a3d6db5cd788ca9ac247a12ea4bc3b3fd81d2bae75b"} Jun 02 05:36:16.796886 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:16.796844 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-649d57b67d-4tp6b" event={"ID":"bab5d774-5c5d-4bbc-85a9-a2c16961df4b","Type":"ContainerStarted","Data":"d6e016d7d84cab8d109293c939509210533c0b360e5de0ffe77d7468ab121f3b"} Jun 02 05:36:16.796886 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:16.796894 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:16.816115 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:16.816066 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-649d57b67d-4tp6b" podStartSLOduration=2.128251128 podStartE2EDuration="3.816050428s" podCreationTimestamp="2026-06-02 05:36:13 +0000 UTC" firstStartedPulling="2026-06-02 05:36:14.103322901 +0000 UTC m=+625.813772126" lastFinishedPulling="2026-06-02 05:36:15.791122202 +0000 UTC m=+627.501571426" observedRunningTime="2026-06-02 05:36:16.815009236 +0000 UTC m=+628.525458478" watchObservedRunningTime="2026-06-02 05:36:16.816050428 +0000 UTC m=+628.526499675" Jun 02 05:36:22.806456 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:22.806424 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-649d57b67d-4tp6b" Jun 02 05:36:24.696244 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:24.696202 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:36:24.696665 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:24.696461 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" containerID="cri-o://90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4" gracePeriod=30 Jun 02 05:36:26.736254 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.736229 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:26.781789 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.781752 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ld5s\" (UniqueName: \"kubernetes.io/projected/a96059bd-4e1f-40b8-b6de-8e8b657fc43a-kube-api-access-8ld5s\") pod \"a96059bd-4e1f-40b8-b6de-8e8b657fc43a\" (UID: \"a96059bd-4e1f-40b8-b6de-8e8b657fc43a\") " Jun 02 05:36:26.784274 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.784246 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a96059bd-4e1f-40b8-b6de-8e8b657fc43a-kube-api-access-8ld5s" (OuterVolumeSpecName: "kube-api-access-8ld5s") pod "a96059bd-4e1f-40b8-b6de-8e8b657fc43a" (UID: "a96059bd-4e1f-40b8-b6de-8e8b657fc43a"). InnerVolumeSpecName "kube-api-access-8ld5s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:36:26.834980 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.834935 2561 generic.go:358] "Generic (PLEG): container finished" podID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerID="90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4" exitCode=143 Jun 02 05:36:26.835090 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.835009 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:26.835090 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.835024 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"a96059bd-4e1f-40b8-b6de-8e8b657fc43a","Type":"ContainerDied","Data":"90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4"} Jun 02 05:36:26.835090 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.835060 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"a96059bd-4e1f-40b8-b6de-8e8b657fc43a","Type":"ContainerDied","Data":"aadb02dcfff94823a82053297d2e92b52631d00d0daa204f5b2af424d4bce96a"} Jun 02 05:36:26.835090 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.835077 2561 scope.go:117] "RemoveContainer" containerID="90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4" Jun 02 05:36:26.843710 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.843692 2561 scope.go:117] "RemoveContainer" containerID="90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4" Jun 02 05:36:26.843953 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:36:26.843933 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4\": container with ID starting with 90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4 not found: ID does not exist" containerID="90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4" Jun 02 05:36:26.844025 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.843979 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4"} err="failed to get container status \"90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4\": rpc error: code = NotFound desc = could not find container \"90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4\": container with ID starting with 90d725a70216d842884f2fe931d6e53cef079d450faa8e3e46d18e4ee72b39e4 not found: ID does not exist" Jun 02 05:36:26.854431 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.854403 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:36:26.858060 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.858043 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:36:26.882772 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.882711 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ld5s\" (UniqueName: \"kubernetes.io/projected/a96059bd-4e1f-40b8-b6de-8e8b657fc43a-kube-api-access-8ld5s\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:36:26.885668 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.885649 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:36:26.886003 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.885990 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" Jun 02 05:36:26.886054 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.886004 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" Jun 02 05:36:26.886097 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.886056 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" containerName="keycloak" Jun 02 05:36:26.890268 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.890253 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:26.892877 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.892852 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 02 05:36:26.893003 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.892914 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 05:36:26.893003 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.892925 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 05:36:26.893003 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.892914 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-xvbfm\"" Jun 02 05:36:26.893003 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.892915 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 05:36:26.898241 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.898217 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:36:26.983893 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.983860 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwjvk\" (UniqueName: \"kubernetes.io/projected/1db5339a-5e16-4b15-b8ce-fde96b8488e7-kube-api-access-nwjvk\") pod \"maas-keycloak-0\" (UID: \"1db5339a-5e16-4b15-b8ce-fde96b8488e7\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:26.984075 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:26.983899 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/1db5339a-5e16-4b15-b8ce-fde96b8488e7-test-realms\") pod \"maas-keycloak-0\" (UID: \"1db5339a-5e16-4b15-b8ce-fde96b8488e7\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:27.084720 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.084686 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nwjvk\" (UniqueName: \"kubernetes.io/projected/1db5339a-5e16-4b15-b8ce-fde96b8488e7-kube-api-access-nwjvk\") pod \"maas-keycloak-0\" (UID: \"1db5339a-5e16-4b15-b8ce-fde96b8488e7\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:27.084875 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.084731 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/1db5339a-5e16-4b15-b8ce-fde96b8488e7-test-realms\") pod \"maas-keycloak-0\" (UID: \"1db5339a-5e16-4b15-b8ce-fde96b8488e7\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:27.085390 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.085372 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/1db5339a-5e16-4b15-b8ce-fde96b8488e7-test-realms\") pod \"maas-keycloak-0\" (UID: \"1db5339a-5e16-4b15-b8ce-fde96b8488e7\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:27.095120 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.095088 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwjvk\" (UniqueName: \"kubernetes.io/projected/1db5339a-5e16-4b15-b8ce-fde96b8488e7-kube-api-access-nwjvk\") pod \"maas-keycloak-0\" (UID: \"1db5339a-5e16-4b15-b8ce-fde96b8488e7\") " pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:27.200510 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.200425 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:27.330890 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.330863 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 05:36:27.332725 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:36:27.332699 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db5339a_5e16_4b15_b8ce_fde96b8488e7.slice/crio-683a715c0d7b24a4ad77838d0ca35332daa5d4d49d0cdde94ceda8d1496e588d WatchSource:0}: Error finding container 683a715c0d7b24a4ad77838d0ca35332daa5d4d49d0cdde94ceda8d1496e588d: Status 404 returned error can't find the container with id 683a715c0d7b24a4ad77838d0ca35332daa5d4d49d0cdde94ceda8d1496e588d Jun 02 05:36:27.840622 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.840592 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"1db5339a-5e16-4b15-b8ce-fde96b8488e7","Type":"ContainerStarted","Data":"9bb811b6cc49153c0498cd8f25834d3a809ee194b0cc0aa071394e1bb1427c86"} Jun 02 05:36:27.840622 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.840625 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"1db5339a-5e16-4b15-b8ce-fde96b8488e7","Type":"ContainerStarted","Data":"683a715c0d7b24a4ad77838d0ca35332daa5d4d49d0cdde94ceda8d1496e588d"} Jun 02 05:36:27.860226 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:27.860184 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.465533387 podStartE2EDuration="1.860170196s" podCreationTimestamp="2026-06-02 05:36:26 +0000 UTC" firstStartedPulling="2026-06-02 05:36:27.334080288 +0000 UTC m=+639.044529511" lastFinishedPulling="2026-06-02 05:36:27.728717083 +0000 UTC m=+639.439166320" observedRunningTime="2026-06-02 05:36:27.859062727 +0000 UTC m=+639.569512004" watchObservedRunningTime="2026-06-02 05:36:27.860170196 +0000 UTC m=+639.570619443" Jun 02 05:36:28.201163 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:28.201121 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:28.202887 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:28.202856 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:28.788942 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:28.788897 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a96059bd-4e1f-40b8-b6de-8e8b657fc43a" path="/var/lib/kubelet/pods/a96059bd-4e1f-40b8-b6de-8e8b657fc43a/volumes" Jun 02 05:36:29.201309 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:29.201197 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:30.201823 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:30.201766 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:31.201832 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:31.201775 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:32.201242 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:32.201191 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:33.201187 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:33.201134 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:34.201858 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:34.201802 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:35.201772 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:35.201716 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:36.201654 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:36.201606 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:37.201011 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:37.200950 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:37.201353 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:37.201316 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:38.201498 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:38.201442 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:39.201171 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:39.201119 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:40.201628 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:40.201578 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" Jun 02 05:36:41.330065 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:41.330024 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 05:36:41.349019 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:41.348948 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="1db5339a-5e16-4b15-b8ce-fde96b8488e7" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 05:36:51.336026 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:36:51.335988 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 05:37:02.086295 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.086256 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d4d89597d-nn2fh"] Jun 02 05:37:02.090917 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.090893 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.093591 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.093571 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 02 05:37:02.100512 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.100488 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d4d89597d-nn2fh"] Jun 02 05:37:02.111435 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.111410 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40f762ce-6f55-47f9-a479-9aa7224c7a53-tls-cert\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.111580 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.111474 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz4qx\" (UniqueName: \"kubernetes.io/projected/40f762ce-6f55-47f9-a479-9aa7224c7a53-kube-api-access-qz4qx\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.111580 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.111507 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/40f762ce-6f55-47f9-a479-9aa7224c7a53-oidc-ca\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.212192 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.212153 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qz4qx\" (UniqueName: \"kubernetes.io/projected/40f762ce-6f55-47f9-a479-9aa7224c7a53-kube-api-access-qz4qx\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.212382 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.212204 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/40f762ce-6f55-47f9-a479-9aa7224c7a53-oidc-ca\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.212382 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.212249 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40f762ce-6f55-47f9-a479-9aa7224c7a53-tls-cert\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.212883 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.212858 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/40f762ce-6f55-47f9-a479-9aa7224c7a53-oidc-ca\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.214698 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.214678 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40f762ce-6f55-47f9-a479-9aa7224c7a53-tls-cert\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.223104 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.223079 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz4qx\" (UniqueName: \"kubernetes.io/projected/40f762ce-6f55-47f9-a479-9aa7224c7a53-kube-api-access-qz4qx\") pod \"authorino-7d4d89597d-nn2fh\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.401018 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.400925 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:37:02.550158 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.550130 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d4d89597d-nn2fh"] Jun 02 05:37:02.552698 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:37:02.552668 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40f762ce_6f55_47f9_a479_9aa7224c7a53.slice/crio-3cf64d445591804333fb189b8720552b2ab402bbef4de35cf5daf2ddb5899921 WatchSource:0}: Error finding container 3cf64d445591804333fb189b8720552b2ab402bbef4de35cf5daf2ddb5899921: Status 404 returned error can't find the container with id 3cf64d445591804333fb189b8720552b2ab402bbef4de35cf5daf2ddb5899921 Jun 02 05:37:02.977406 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:02.977373 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" event={"ID":"40f762ce-6f55-47f9-a479-9aa7224c7a53","Type":"ContainerStarted","Data":"3cf64d445591804333fb189b8720552b2ab402bbef4de35cf5daf2ddb5899921"} Jun 02 05:37:03.982113 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:03.982082 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" event={"ID":"40f762ce-6f55-47f9-a479-9aa7224c7a53","Type":"ContainerStarted","Data":"f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676"} Jun 02 05:37:04.000958 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.000893 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" podStartSLOduration=1.535117437 podStartE2EDuration="2.000879597s" podCreationTimestamp="2026-06-02 05:37:02 +0000 UTC" firstStartedPulling="2026-06-02 05:37:02.553932325 +0000 UTC m=+674.264381551" lastFinishedPulling="2026-06-02 05:37:03.019694483 +0000 UTC m=+674.730143711" observedRunningTime="2026-06-02 05:37:03.999349343 +0000 UTC m=+675.709798613" watchObservedRunningTime="2026-06-02 05:37:04.000879597 +0000 UTC m=+675.711328842" Jun 02 05:37:04.031452 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.031413 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c94cb989d-zsxmg"] Jun 02 05:37:04.031650 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.031608 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-c94cb989d-zsxmg" podUID="1c26742c-bba8-4f7c-98f4-06a0082346ee" containerName="authorino" containerID="cri-o://0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d" gracePeriod=30 Jun 02 05:37:04.281559 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.281538 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:37:04.331533 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.331507 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm67k\" (UniqueName: \"kubernetes.io/projected/1c26742c-bba8-4f7c-98f4-06a0082346ee-kube-api-access-vm67k\") pod \"1c26742c-bba8-4f7c-98f4-06a0082346ee\" (UID: \"1c26742c-bba8-4f7c-98f4-06a0082346ee\") " Jun 02 05:37:04.331688 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.331570 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c26742c-bba8-4f7c-98f4-06a0082346ee-tls-cert\") pod \"1c26742c-bba8-4f7c-98f4-06a0082346ee\" (UID: \"1c26742c-bba8-4f7c-98f4-06a0082346ee\") " Jun 02 05:37:04.333717 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.333689 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c26742c-bba8-4f7c-98f4-06a0082346ee-kube-api-access-vm67k" (OuterVolumeSpecName: "kube-api-access-vm67k") pod "1c26742c-bba8-4f7c-98f4-06a0082346ee" (UID: "1c26742c-bba8-4f7c-98f4-06a0082346ee"). InnerVolumeSpecName "kube-api-access-vm67k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:37:04.341627 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.341604 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c26742c-bba8-4f7c-98f4-06a0082346ee-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1c26742c-bba8-4f7c-98f4-06a0082346ee" (UID: "1c26742c-bba8-4f7c-98f4-06a0082346ee"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 05:37:04.432347 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.432319 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm67k\" (UniqueName: \"kubernetes.io/projected/1c26742c-bba8-4f7c-98f4-06a0082346ee-kube-api-access-vm67k\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:37:04.432347 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.432347 2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c26742c-bba8-4f7c-98f4-06a0082346ee-tls-cert\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:37:04.986758 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.986717 2561 generic.go:358] "Generic (PLEG): container finished" podID="1c26742c-bba8-4f7c-98f4-06a0082346ee" containerID="0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d" exitCode=0 Jun 02 05:37:04.987223 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.986765 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c94cb989d-zsxmg" Jun 02 05:37:04.987223 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.986799 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c94cb989d-zsxmg" event={"ID":"1c26742c-bba8-4f7c-98f4-06a0082346ee","Type":"ContainerDied","Data":"0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d"} Jun 02 05:37:04.987223 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.986835 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c94cb989d-zsxmg" event={"ID":"1c26742c-bba8-4f7c-98f4-06a0082346ee","Type":"ContainerDied","Data":"481e41385f2f537d24bf056b8081abea3c46f285f35abff82d1814996f1af04b"} Jun 02 05:37:04.987223 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.986850 2561 scope.go:117] "RemoveContainer" containerID="0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d" Jun 02 05:37:04.995006 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.994987 2561 scope.go:117] "RemoveContainer" containerID="0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d" Jun 02 05:37:04.995264 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:37:04.995245 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d\": container with ID starting with 0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d not found: ID does not exist" containerID="0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d" Jun 02 05:37:04.995327 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:04.995272 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d"} err="failed to get container status \"0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d\": rpc error: code = NotFound desc = could not find container \"0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d\": container with ID starting with 0ae4eea12907f98906b3b23d7b5e1bb150e0e47b450e73f0839463754fb8d22d not found: ID does not exist" Jun 02 05:37:05.001939 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:05.001919 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c94cb989d-zsxmg"] Jun 02 05:37:05.008449 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:05.008427 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c94cb989d-zsxmg"] Jun 02 05:37:06.786866 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:06.786834 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c26742c-bba8-4f7c-98f4-06a0082346ee" path="/var/lib/kubelet/pods/1c26742c-bba8-4f7c-98f4-06a0082346ee/volumes" Jun 02 05:37:08.917846 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.917815 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb"] Jun 02 05:37:08.918363 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.918344 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c26742c-bba8-4f7c-98f4-06a0082346ee" containerName="authorino" Jun 02 05:37:08.918428 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.918368 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c26742c-bba8-4f7c-98f4-06a0082346ee" containerName="authorino" Jun 02 05:37:08.918489 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.918447 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="1c26742c-bba8-4f7c-98f4-06a0082346ee" containerName="authorino" Jun 02 05:37:08.921421 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.921400 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:08.924223 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.924201 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 05:37:08.925180 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.925162 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 05:37:08.925254 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.925176 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 02 05:37:08.925301 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.925272 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cdgqj\"" Jun 02 05:37:08.936500 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.936476 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb"] Jun 02 05:37:08.974888 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.974860 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:08.975034 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.974907 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/111e75a4-b829-4b45-b6b4-970de5fd7c14-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:08.975034 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.975002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7zjd\" (UniqueName: \"kubernetes.io/projected/111e75a4-b829-4b45-b6b4-970de5fd7c14-kube-api-access-l7zjd\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:08.975153 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.975077 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:08.975153 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.975117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:08.975212 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:08.975163 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076388 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076347 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076579 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076419 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076579 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076453 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/111e75a4-b829-4b45-b6b4-970de5fd7c14-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076579 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076493 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l7zjd\" (UniqueName: \"kubernetes.io/projected/111e75a4-b829-4b45-b6b4-970de5fd7c14-kube-api-access-l7zjd\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076579 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076551 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076802 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076597 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076802 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076781 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076904 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076836 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.076955 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.076907 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.078873 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.078850 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/111e75a4-b829-4b45-b6b4-970de5fd7c14-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.079169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.079154 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/111e75a4-b829-4b45-b6b4-970de5fd7c14-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.085158 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.085141 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7zjd\" (UniqueName: \"kubernetes.io/projected/111e75a4-b829-4b45-b6b4-970de5fd7c14-kube-api-access-l7zjd\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb\" (UID: \"111e75a4-b829-4b45-b6b4-970de5fd7c14\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.231770 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.231688 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:09.362647 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:09.362611 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb"] Jun 02 05:37:09.366519 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:37:09.366489 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111e75a4_b829_4b45_b6b4_970de5fd7c14.slice/crio-e027fba190d1e71f1740e1b42a75efa868775790622d9b5057f34bfa6634a577 WatchSource:0}: Error finding container e027fba190d1e71f1740e1b42a75efa868775790622d9b5057f34bfa6634a577: Status 404 returned error can't find the container with id e027fba190d1e71f1740e1b42a75efa868775790622d9b5057f34bfa6634a577 Jun 02 05:37:10.005809 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:10.005773 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" event={"ID":"111e75a4-b829-4b45-b6b4-970de5fd7c14","Type":"ContainerStarted","Data":"e027fba190d1e71f1740e1b42a75efa868775790622d9b5057f34bfa6634a577"} Jun 02 05:37:14.020909 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:14.020872 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" event={"ID":"111e75a4-b829-4b45-b6b4-970de5fd7c14","Type":"ContainerStarted","Data":"6e9fc08a1ec56b4d226a17445f53c3cd17e3d8dff90802d84557035841ef5f9b"} Jun 02 05:37:15.031637 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:15.031546 2561 generic.go:358] "Generic (PLEG): container finished" podID="111e75a4-b829-4b45-b6b4-970de5fd7c14" containerID="6e9fc08a1ec56b4d226a17445f53c3cd17e3d8dff90802d84557035841ef5f9b" exitCode=0 Jun 02 05:37:15.032028 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:15.031625 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" event={"ID":"111e75a4-b829-4b45-b6b4-970de5fd7c14","Type":"ContainerDied","Data":"6e9fc08a1ec56b4d226a17445f53c3cd17e3d8dff90802d84557035841ef5f9b"} Jun 02 05:37:17.040922 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:17.040884 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" event={"ID":"111e75a4-b829-4b45-b6b4-970de5fd7c14","Type":"ContainerStarted","Data":"7692e61af2240a4570526d0cd68fd43775417b8de636d3cd352ea4a66777e49c"} Jun 02 05:37:17.061862 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:17.061816 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" podStartSLOduration=2.325319872 podStartE2EDuration="9.061801932s" podCreationTimestamp="2026-06-02 05:37:08 +0000 UTC" firstStartedPulling="2026-06-02 05:37:09.371017599 +0000 UTC m=+681.081466822" lastFinishedPulling="2026-06-02 05:37:16.107499644 +0000 UTC m=+687.817948882" observedRunningTime="2026-06-02 05:37:17.060790382 +0000 UTC m=+688.771239640" watchObservedRunningTime="2026-06-02 05:37:17.061801932 +0000 UTC m=+688.772251177" Jun 02 05:37:19.232838 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:19.232802 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:19.232838 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:19.232843 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:19.245370 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:19.245349 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:20.063299 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:20.063268 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb" Jun 02 05:37:21.508573 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.508543 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh"] Jun 02 05:37:21.513512 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.513494 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.516031 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.516010 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 02 05:37:21.525746 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.525726 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh"] Jun 02 05:37:21.591787 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.591750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.592004 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.591798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.592004 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.591914 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.592004 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.591990 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fefe6d3b-6c83-4031-85b2-fa842ca362e5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.592167 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.592028 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.592167 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.592063 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55z7q\" (UniqueName: \"kubernetes.io/projected/fefe6d3b-6c83-4031-85b2-fa842ca362e5-kube-api-access-55z7q\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.693492 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.693458 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.693686 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.693504 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.693686 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.693548 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.693686 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.693574 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fefe6d3b-6c83-4031-85b2-fa842ca362e5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.693686 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.693595 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.693686 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.693617 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-55z7q\" (UniqueName: \"kubernetes.io/projected/fefe6d3b-6c83-4031-85b2-fa842ca362e5-kube-api-access-55z7q\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.693950 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.693930 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.694035 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.693990 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.694035 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.694012 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.696046 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.696027 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fefe6d3b-6c83-4031-85b2-fa842ca362e5-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.696310 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.696290 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fefe6d3b-6c83-4031-85b2-fa842ca362e5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.701405 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.701381 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55z7q\" (UniqueName: \"kubernetes.io/projected/fefe6d3b-6c83-4031-85b2-fa842ca362e5-kube-api-access-55z7q\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh\" (UID: \"fefe6d3b-6c83-4031-85b2-fa842ca362e5\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.823399 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.823370 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:21.955015 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:21.954986 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh"] Jun 02 05:37:21.957668 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:37:21.957632 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefe6d3b_6c83_4031_85b2_fa842ca362e5.slice/crio-870b7c551c0fc856173e51f601b9e9b73d39dd8f20151c4037ef3273551ac447 WatchSource:0}: Error finding container 870b7c551c0fc856173e51f601b9e9b73d39dd8f20151c4037ef3273551ac447: Status 404 returned error can't find the container with id 870b7c551c0fc856173e51f601b9e9b73d39dd8f20151c4037ef3273551ac447 Jun 02 05:37:22.059860 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:22.059823 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" event={"ID":"fefe6d3b-6c83-4031-85b2-fa842ca362e5","Type":"ContainerStarted","Data":"10023c97f031ad53d58f6048a5ebea0368827a2a2c822057fef38fbe70e19657"} Jun 02 05:37:22.060043 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:22.059870 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" event={"ID":"fefe6d3b-6c83-4031-85b2-fa842ca362e5","Type":"ContainerStarted","Data":"870b7c551c0fc856173e51f601b9e9b73d39dd8f20151c4037ef3273551ac447"} Jun 02 05:37:24.067839 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:24.067804 2561 generic.go:358] "Generic (PLEG): container finished" podID="fefe6d3b-6c83-4031-85b2-fa842ca362e5" containerID="10023c97f031ad53d58f6048a5ebea0368827a2a2c822057fef38fbe70e19657" exitCode=0 Jun 02 05:37:24.068332 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:24.067861 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" event={"ID":"fefe6d3b-6c83-4031-85b2-fa842ca362e5","Type":"ContainerDied","Data":"10023c97f031ad53d58f6048a5ebea0368827a2a2c822057fef38fbe70e19657"} Jun 02 05:37:25.072680 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:25.072646 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" event={"ID":"fefe6d3b-6c83-4031-85b2-fa842ca362e5","Type":"ContainerStarted","Data":"29a852b4dcbb15d4b92a6037acbc53fab05f199116665a6ef5db4aa1fd6948e5"} Jun 02 05:37:25.099525 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:25.099483 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" podStartSLOduration=3.906943151 podStartE2EDuration="4.099470155s" podCreationTimestamp="2026-06-02 05:37:21 +0000 UTC" firstStartedPulling="2026-06-02 05:37:24.068629332 +0000 UTC m=+695.779078557" lastFinishedPulling="2026-06-02 05:37:24.261156333 +0000 UTC m=+695.971605561" observedRunningTime="2026-06-02 05:37:25.098118789 +0000 UTC m=+696.808568036" watchObservedRunningTime="2026-06-02 05:37:25.099470155 +0000 UTC m=+696.809919401" Jun 02 05:37:31.823891 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:31.823856 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:31.824445 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:31.824015 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:31.836494 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:31.836469 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:32.105978 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:32.105886 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh" Jun 02 05:37:39.026886 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.026807 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7"] Jun 02 05:37:39.030411 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.030391 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.032897 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.032877 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 02 05:37:39.041279 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.041253 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7"] Jun 02 05:37:39.050219 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.050194 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.050333 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.050241 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.050333 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.050258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c7e1ee-aaab-433b-b09a-189131fe636c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.050333 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.050289 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.050461 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.050356 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.050461 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.050394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljpxv\" (UniqueName: \"kubernetes.io/projected/87c7e1ee-aaab-433b-b09a-189131fe636c-kube-api-access-ljpxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.151634 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.151603 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.151815 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.151648 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.151815 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.151665 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c7e1ee-aaab-433b-b09a-189131fe636c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.151815 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.151688 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.151815 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.151726 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.151815 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.151757 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ljpxv\" (UniqueName: \"kubernetes.io/projected/87c7e1ee-aaab-433b-b09a-189131fe636c-kube-api-access-ljpxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.152110 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.152087 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.152168 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.152113 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.152226 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.152205 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.154232 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.154208 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87c7e1ee-aaab-433b-b09a-189131fe636c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.154350 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.154335 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c7e1ee-aaab-433b-b09a-189131fe636c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.159948 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.159927 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljpxv\" (UniqueName: \"kubernetes.io/projected/87c7e1ee-aaab-433b-b09a-189131fe636c-kube-api-access-ljpxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7\" (UID: \"87c7e1ee-aaab-433b-b09a-189131fe636c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.341325 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.341283 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:39.473359 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:39.473332 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7"] Jun 02 05:37:39.475766 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:37:39.475737 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c7e1ee_aaab_433b_b09a_189131fe636c.slice/crio-5bdc12c18e8698552b66969b6aa3b53e91d24880df25d9c0b034b3bb324d2aed WatchSource:0}: Error finding container 5bdc12c18e8698552b66969b6aa3b53e91d24880df25d9c0b034b3bb324d2aed: Status 404 returned error can't find the container with id 5bdc12c18e8698552b66969b6aa3b53e91d24880df25d9c0b034b3bb324d2aed Jun 02 05:37:40.121842 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:40.121796 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" event={"ID":"87c7e1ee-aaab-433b-b09a-189131fe636c","Type":"ContainerStarted","Data":"e4ba6ab0780e9923e9a34ccc248e1e1e475be46a7c704d6d7776632b624cc82e"} Jun 02 05:37:40.121842 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:40.121838 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" event={"ID":"87c7e1ee-aaab-433b-b09a-189131fe636c","Type":"ContainerStarted","Data":"5bdc12c18e8698552b66969b6aa3b53e91d24880df25d9c0b034b3bb324d2aed"} Jun 02 05:37:41.126285 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:41.126247 2561 generic.go:358] "Generic (PLEG): container finished" podID="87c7e1ee-aaab-433b-b09a-189131fe636c" containerID="e4ba6ab0780e9923e9a34ccc248e1e1e475be46a7c704d6d7776632b624cc82e" exitCode=0 Jun 02 05:37:41.126699 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:41.126340 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" event={"ID":"87c7e1ee-aaab-433b-b09a-189131fe636c","Type":"ContainerDied","Data":"e4ba6ab0780e9923e9a34ccc248e1e1e475be46a7c704d6d7776632b624cc82e"} Jun 02 05:37:42.131293 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:42.131257 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" event={"ID":"87c7e1ee-aaab-433b-b09a-189131fe636c","Type":"ContainerStarted","Data":"e3a673d0adc740c6cb102770e119caf31d65c5815d5dd817e412c7050517ef03"} Jun 02 05:37:42.156016 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:42.155948 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" podStartSLOduration=2.94192284 podStartE2EDuration="3.155932912s" podCreationTimestamp="2026-06-02 05:37:39 +0000 UTC" firstStartedPulling="2026-06-02 05:37:41.12684426 +0000 UTC m=+712.837293484" lastFinishedPulling="2026-06-02 05:37:41.340854328 +0000 UTC m=+713.051303556" observedRunningTime="2026-06-02 05:37:42.153364705 +0000 UTC m=+713.863813976" watchObservedRunningTime="2026-06-02 05:37:42.155932912 +0000 UTC m=+713.866382196" Jun 02 05:37:49.341886 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:49.341848 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:49.342327 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:49.341920 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:49.354699 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:49.354671 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:50.169715 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.169685 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7" Jun 02 05:37:50.519993 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.519887 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8"] Jun 02 05:37:50.525884 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.525864 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.528293 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.528276 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 02 05:37:50.534949 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.534928 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8"] Jun 02 05:37:50.554008 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.553981 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.554118 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.554047 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d62f521f-295a-4789-a92b-1435f293bd1f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.554118 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.554076 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.554118 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.554093 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.554234 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.554166 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psv8t\" (UniqueName: \"kubernetes.io/projected/d62f521f-295a-4789-a92b-1435f293bd1f-kube-api-access-psv8t\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.554234 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.554215 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.654709 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.654677 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.654834 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.654726 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d62f521f-295a-4789-a92b-1435f293bd1f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.654834 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.654755 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.654834 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.654784 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.654834 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.654821 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-psv8t\" (UniqueName: \"kubernetes.io/projected/d62f521f-295a-4789-a92b-1435f293bd1f-kube-api-access-psv8t\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.655002 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.654867 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.655188 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.655156 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.655265 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.655190 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.655319 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.655282 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.657216 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.657195 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d62f521f-295a-4789-a92b-1435f293bd1f-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.657440 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.657423 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d62f521f-295a-4789-a92b-1435f293bd1f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.663017 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.662985 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psv8t\" (UniqueName: \"kubernetes.io/projected/d62f521f-295a-4789-a92b-1435f293bd1f-kube-api-access-psv8t\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8\" (UID: \"d62f521f-295a-4789-a92b-1435f293bd1f\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.836504 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.836471 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:37:50.975464 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:50.975430 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8"] Jun 02 05:37:50.990167 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:37:50.990138 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62f521f_295a_4789_a92b_1435f293bd1f.slice/crio-a4b02eb0499cc6c889f54bfdf672f18186c44dea6eb2cd81bce83b6a5adecd54 WatchSource:0}: Error finding container a4b02eb0499cc6c889f54bfdf672f18186c44dea6eb2cd81bce83b6a5adecd54: Status 404 returned error can't find the container with id a4b02eb0499cc6c889f54bfdf672f18186c44dea6eb2cd81bce83b6a5adecd54 Jun 02 05:37:51.166763 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:51.166668 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" event={"ID":"d62f521f-295a-4789-a92b-1435f293bd1f","Type":"ContainerStarted","Data":"8fddbf888397e67d79e47832f2e2ebbf6b074e9c3bae547f76b887d9ba70320b"} Jun 02 05:37:51.166763 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:37:51.166715 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" event={"ID":"d62f521f-295a-4789-a92b-1435f293bd1f","Type":"ContainerStarted","Data":"a4b02eb0499cc6c889f54bfdf672f18186c44dea6eb2cd81bce83b6a5adecd54"} Jun 02 05:38:43.341882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:38:43.341844 2561 generic.go:358] "Generic (PLEG): container finished" podID="d62f521f-295a-4789-a92b-1435f293bd1f" containerID="8fddbf888397e67d79e47832f2e2ebbf6b074e9c3bae547f76b887d9ba70320b" exitCode=0 Jun 02 05:38:43.341882 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:38:43.341881 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" event={"ID":"d62f521f-295a-4789-a92b-1435f293bd1f","Type":"ContainerDied","Data":"8fddbf888397e67d79e47832f2e2ebbf6b074e9c3bae547f76b887d9ba70320b"} Jun 02 05:38:44.346635 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:38:44.346596 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" event={"ID":"d62f521f-295a-4789-a92b-1435f293bd1f","Type":"ContainerStarted","Data":"01d5c09ffbbe9b9f481050a8ff3d3a8d64b18463a2aa6c0fa3b5e28e1edd8f5b"} Jun 02 05:38:44.369183 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:38:44.369130 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" podStartSLOduration=54.198585533 podStartE2EDuration="54.369112967s" podCreationTimestamp="2026-06-02 05:37:50 +0000 UTC" firstStartedPulling="2026-06-02 05:38:43.342510941 +0000 UTC m=+775.052960165" lastFinishedPulling="2026-06-02 05:38:43.513038372 +0000 UTC m=+775.223487599" observedRunningTime="2026-06-02 05:38:44.366761567 +0000 UTC m=+776.077210813" watchObservedRunningTime="2026-06-02 05:38:44.369112967 +0000 UTC m=+776.079562215" Jun 02 05:38:50.836648 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:38:50.836612 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:38:50.836648 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:38:50.836649 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:38:50.849098 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:38:50.849073 2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:38:51.380956 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:38:51.380926 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8" Jun 02 05:39:20.011759 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.011678 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-668ccb6567-ssdv2"] Jun 02 05:39:20.015089 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.015071 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.021504 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.021479 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-668ccb6567-ssdv2"] Jun 02 05:39:20.049819 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.049791 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66760c9c-b6f0-4f04-b674-a693b716c85b-tls-cert\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.049981 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.049849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/66760c9c-b6f0-4f04-b674-a693b716c85b-oidc-ca\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.049981 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.049870 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ss9w\" (UniqueName: \"kubernetes.io/projected/66760c9c-b6f0-4f04-b674-a693b716c85b-kube-api-access-5ss9w\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.150745 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.150713 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66760c9c-b6f0-4f04-b674-a693b716c85b-tls-cert\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.150952 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.150766 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/66760c9c-b6f0-4f04-b674-a693b716c85b-oidc-ca\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.150952 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.150785 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5ss9w\" (UniqueName: \"kubernetes.io/projected/66760c9c-b6f0-4f04-b674-a693b716c85b-kube-api-access-5ss9w\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.151389 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.151368 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/66760c9c-b6f0-4f04-b674-a693b716c85b-oidc-ca\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.153260 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.153239 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66760c9c-b6f0-4f04-b674-a693b716c85b-tls-cert\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.158360 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.158332 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ss9w\" (UniqueName: \"kubernetes.io/projected/66760c9c-b6f0-4f04-b674-a693b716c85b-kube-api-access-5ss9w\") pod \"authorino-668ccb6567-ssdv2\" (UID: \"66760c9c-b6f0-4f04-b674-a693b716c85b\") " pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.325047 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.325018 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-668ccb6567-ssdv2" Jun 02 05:39:20.450595 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.450556 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-668ccb6567-ssdv2"] Jun 02 05:39:20.453440 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:39:20.453407 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66760c9c_b6f0_4f04_b674_a693b716c85b.slice/crio-04589392c76cbe29ac1a4405a8d6d91eadbdcf22916e5a6fc09c1e8af7b82fea WatchSource:0}: Error finding container 04589392c76cbe29ac1a4405a8d6d91eadbdcf22916e5a6fc09c1e8af7b82fea: Status 404 returned error can't find the container with id 04589392c76cbe29ac1a4405a8d6d91eadbdcf22916e5a6fc09c1e8af7b82fea Jun 02 05:39:20.464672 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:20.464645 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-668ccb6567-ssdv2" event={"ID":"66760c9c-b6f0-4f04-b674-a693b716c85b","Type":"ContainerStarted","Data":"04589392c76cbe29ac1a4405a8d6d91eadbdcf22916e5a6fc09c1e8af7b82fea"} Jun 02 05:39:21.471048 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.471011 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-668ccb6567-ssdv2" event={"ID":"66760c9c-b6f0-4f04-b674-a693b716c85b","Type":"ContainerStarted","Data":"1a072b1a377385f2f03ec8bd379dfe5e82bfabf0473c53174b6498e93ddbd592"} Jun 02 05:39:21.490256 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.490191 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-668ccb6567-ssdv2" podStartSLOduration=2.095273833 podStartE2EDuration="2.49017118s" podCreationTimestamp="2026-06-02 05:39:19 +0000 UTC" firstStartedPulling="2026-06-02 05:39:20.45480189 +0000 UTC m=+812.165251113" lastFinishedPulling="2026-06-02 05:39:20.849699236 +0000 UTC m=+812.560148460" observedRunningTime="2026-06-02 05:39:21.488292703 +0000 UTC m=+813.198741951" watchObservedRunningTime="2026-06-02 05:39:21.49017118 +0000 UTC m=+813.200620427" Jun 02 05:39:21.516507 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.516462 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d4d89597d-nn2fh"] Jun 02 05:39:21.516773 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.516739 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" podUID="40f762ce-6f55-47f9-a479-9aa7224c7a53" containerName="authorino" containerID="cri-o://f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676" gracePeriod=30 Jun 02 05:39:21.777161 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.777139 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:39:21.866753 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.866721 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz4qx\" (UniqueName: \"kubernetes.io/projected/40f762ce-6f55-47f9-a479-9aa7224c7a53-kube-api-access-qz4qx\") pod \"40f762ce-6f55-47f9-a479-9aa7224c7a53\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " Jun 02 05:39:21.866910 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.866771 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40f762ce-6f55-47f9-a479-9aa7224c7a53-tls-cert\") pod \"40f762ce-6f55-47f9-a479-9aa7224c7a53\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " Jun 02 05:39:21.866910 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.866796 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/40f762ce-6f55-47f9-a479-9aa7224c7a53-oidc-ca\") pod \"40f762ce-6f55-47f9-a479-9aa7224c7a53\" (UID: \"40f762ce-6f55-47f9-a479-9aa7224c7a53\") " Jun 02 05:39:21.869466 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.869433 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f762ce-6f55-47f9-a479-9aa7224c7a53-kube-api-access-qz4qx" (OuterVolumeSpecName: "kube-api-access-qz4qx") pod "40f762ce-6f55-47f9-a479-9aa7224c7a53" (UID: "40f762ce-6f55-47f9-a479-9aa7224c7a53"). InnerVolumeSpecName "kube-api-access-qz4qx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:39:21.871987 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.871936 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40f762ce-6f55-47f9-a479-9aa7224c7a53-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "40f762ce-6f55-47f9-a479-9aa7224c7a53" (UID: "40f762ce-6f55-47f9-a479-9aa7224c7a53"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 05:39:21.877348 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.877327 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40f762ce-6f55-47f9-a479-9aa7224c7a53-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "40f762ce-6f55-47f9-a479-9aa7224c7a53" (UID: "40f762ce-6f55-47f9-a479-9aa7224c7a53"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 05:39:21.967777 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.967738 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz4qx\" (UniqueName: \"kubernetes.io/projected/40f762ce-6f55-47f9-a479-9aa7224c7a53-kube-api-access-qz4qx\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:39:21.967777 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.967771 2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40f762ce-6f55-47f9-a479-9aa7224c7a53-tls-cert\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:39:21.967994 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:21.967786 2561 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/40f762ce-6f55-47f9-a479-9aa7224c7a53-oidc-ca\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:39:22.475446 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.475366 2561 generic.go:358] "Generic (PLEG): container finished" podID="40f762ce-6f55-47f9-a479-9aa7224c7a53" containerID="f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676" exitCode=0 Jun 02 05:39:22.475446 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.475420 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" Jun 02 05:39:22.475905 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.475450 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" event={"ID":"40f762ce-6f55-47f9-a479-9aa7224c7a53","Type":"ContainerDied","Data":"f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676"} Jun 02 05:39:22.475905 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.475482 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d4d89597d-nn2fh" event={"ID":"40f762ce-6f55-47f9-a479-9aa7224c7a53","Type":"ContainerDied","Data":"3cf64d445591804333fb189b8720552b2ab402bbef4de35cf5daf2ddb5899921"} Jun 02 05:39:22.475905 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.475497 2561 scope.go:117] "RemoveContainer" containerID="f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676" Jun 02 05:39:22.484808 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.484792 2561 scope.go:117] "RemoveContainer" containerID="f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676" Jun 02 05:39:22.485087 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:39:22.485067 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676\": container with ID starting with f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676 not found: ID does not exist" containerID="f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676" Jun 02 05:39:22.485175 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.485102 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676"} err="failed to get container status \"f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676\": rpc error: code = NotFound desc = could not find container \"f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676\": container with ID starting with f3ab27f2e28362f2f8140753fbb53b612cc8cdce54433ebd7911c189a4e2f676 not found: ID does not exist" Jun 02 05:39:22.496055 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.496008 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d4d89597d-nn2fh"] Jun 02 05:39:22.497758 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.497739 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d4d89597d-nn2fh"] Jun 02 05:39:22.786891 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:39:22.786816 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40f762ce-6f55-47f9-a479-9aa7224c7a53" path="/var/lib/kubelet/pods/40f762ce-6f55-47f9-a479-9aa7224c7a53/volumes" Jun 02 05:40:48.741688 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:48.741646 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-5fb4m"] Jun 02 05:40:48.742210 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:48.741950 2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" podUID="5c998e38-4090-4948-b86e-c51e00aeb74d" containerName="manager" containerID="cri-o://1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030" gracePeriod=30 Jun 02 05:40:48.988251 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:48.988227 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:40:49.059992 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.059944 2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6qwt\" (UniqueName: \"kubernetes.io/projected/5c998e38-4090-4948-b86e-c51e00aeb74d-kube-api-access-l6qwt\") pod \"5c998e38-4090-4948-b86e-c51e00aeb74d\" (UID: \"5c998e38-4090-4948-b86e-c51e00aeb74d\") " Jun 02 05:40:49.062177 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.062150 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c998e38-4090-4948-b86e-c51e00aeb74d-kube-api-access-l6qwt" (OuterVolumeSpecName: "kube-api-access-l6qwt") pod "5c998e38-4090-4948-b86e-c51e00aeb74d" (UID: "5c998e38-4090-4948-b86e-c51e00aeb74d"). InnerVolumeSpecName "kube-api-access-l6qwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:40:49.160604 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.160576 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6qwt\" (UniqueName: \"kubernetes.io/projected/5c998e38-4090-4948-b86e-c51e00aeb74d-kube-api-access-l6qwt\") on node \"ip-10-0-141-22.ec2.internal\" DevicePath \"\"" Jun 02 05:40:49.783694 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.783661 2561 generic.go:358] "Generic (PLEG): container finished" podID="5c998e38-4090-4948-b86e-c51e00aeb74d" containerID="1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030" exitCode=0 Jun 02 05:40:49.783694 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.783697 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" event={"ID":"5c998e38-4090-4948-b86e-c51e00aeb74d","Type":"ContainerDied","Data":"1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030"} Jun 02 05:40:49.784169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.783715 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" event={"ID":"5c998e38-4090-4948-b86e-c51e00aeb74d","Type":"ContainerDied","Data":"f57790f1ec24a8ba3fbfc1eab16f63616117eb18d285f40d6ad6325459d870f7"} Jun 02 05:40:49.784169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.783715 2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bc6dfc4b6-5fb4m" Jun 02 05:40:49.784169 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.783728 2561 scope.go:117] "RemoveContainer" containerID="1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030" Jun 02 05:40:49.792630 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.792609 2561 scope.go:117] "RemoveContainer" containerID="1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030" Jun 02 05:40:49.792856 ip-10-0-141-22 kubenswrapper[2561]: E0602 05:40:49.792838 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030\": container with ID starting with 1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030 not found: ID does not exist" containerID="1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030" Jun 02 05:40:49.792915 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.792863 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030"} err="failed to get container status \"1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030\": rpc error: code = NotFound desc = could not find container \"1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030\": container with ID starting with 1a85eb8233bf873b0058a34563def9a215f19f82b60eb1a4a822dd86ce834030 not found: ID does not exist" Jun 02 05:40:49.809251 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.809231 2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-5fb4m"] Jun 02 05:40:49.815874 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:49.815854 2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-5fb4m"] Jun 02 05:40:50.575299 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.575261 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-6jn8w"] Jun 02 05:40:50.575611 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.575599 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40f762ce-6f55-47f9-a479-9aa7224c7a53" containerName="authorino" Jun 02 05:40:50.575654 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.575613 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40f762ce-6f55-47f9-a479-9aa7224c7a53" containerName="authorino" Jun 02 05:40:50.575654 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.575626 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c998e38-4090-4948-b86e-c51e00aeb74d" containerName="manager" Jun 02 05:40:50.575654 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.575631 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c998e38-4090-4948-b86e-c51e00aeb74d" containerName="manager" Jun 02 05:40:50.575748 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.575696 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="5c998e38-4090-4948-b86e-c51e00aeb74d" containerName="manager" Jun 02 05:40:50.575748 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.575708 2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="40f762ce-6f55-47f9-a479-9aa7224c7a53" containerName="authorino" Jun 02 05:40:50.578547 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.578528 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" Jun 02 05:40:50.581151 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.581129 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m2qd5\"" Jun 02 05:40:50.581213 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.581154 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 05:40:50.589243 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.589218 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-6jn8w"] Jun 02 05:40:50.674605 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.674570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8clgz\" (UniqueName: \"kubernetes.io/projected/bca4a681-0d98-4a1e-86cf-df35d4ed6cc3-kube-api-access-8clgz\") pod \"maas-controller-7bc6dfc4b6-6jn8w\" (UID: \"bca4a681-0d98-4a1e-86cf-df35d4ed6cc3\") " pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" Jun 02 05:40:50.776080 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.776046 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8clgz\" (UniqueName: \"kubernetes.io/projected/bca4a681-0d98-4a1e-86cf-df35d4ed6cc3-kube-api-access-8clgz\") pod \"maas-controller-7bc6dfc4b6-6jn8w\" (UID: \"bca4a681-0d98-4a1e-86cf-df35d4ed6cc3\") " pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" Jun 02 05:40:50.784376 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.784350 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8clgz\" (UniqueName: \"kubernetes.io/projected/bca4a681-0d98-4a1e-86cf-df35d4ed6cc3-kube-api-access-8clgz\") pod \"maas-controller-7bc6dfc4b6-6jn8w\" (UID: \"bca4a681-0d98-4a1e-86cf-df35d4ed6cc3\") " pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" Jun 02 05:40:50.786449 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.786424 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c998e38-4090-4948-b86e-c51e00aeb74d" path="/var/lib/kubelet/pods/5c998e38-4090-4948-b86e-c51e00aeb74d/volumes" Jun 02 05:40:50.888573 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:50.888500 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" Jun 02 05:40:51.018039 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:51.018015 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bc6dfc4b6-6jn8w"] Jun 02 05:40:51.020104 ip-10-0-141-22 kubenswrapper[2561]: W0602 05:40:51.020076 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca4a681_0d98_4a1e_86cf_df35d4ed6cc3.slice/crio-883337d64992d010c887cbe92e21d241dda73c273b07a49a97da8ac692155219 WatchSource:0}: Error finding container 883337d64992d010c887cbe92e21d241dda73c273b07a49a97da8ac692155219: Status 404 returned error can't find the container with id 883337d64992d010c887cbe92e21d241dda73c273b07a49a97da8ac692155219 Jun 02 05:40:51.792701 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:51.792672 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" event={"ID":"bca4a681-0d98-4a1e-86cf-df35d4ed6cc3","Type":"ContainerStarted","Data":"786042dbf8abc364fb87273eece1235699a194ceca4e2ecccc364c4481efcb2f"} Jun 02 05:40:51.793068 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:51.792710 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" event={"ID":"bca4a681-0d98-4a1e-86cf-df35d4ed6cc3","Type":"ContainerStarted","Data":"883337d64992d010c887cbe92e21d241dda73c273b07a49a97da8ac692155219"} Jun 02 05:40:51.793068 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:51.792736 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" Jun 02 05:40:51.811136 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:40:51.811090 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" podStartSLOduration=1.304634624 podStartE2EDuration="1.811076234s" podCreationTimestamp="2026-06-02 05:40:50 +0000 UTC" firstStartedPulling="2026-06-02 05:40:51.021730462 +0000 UTC m=+902.732179686" lastFinishedPulling="2026-06-02 05:40:51.528172068 +0000 UTC m=+903.238621296" observedRunningTime="2026-06-02 05:40:51.809128696 +0000 UTC m=+903.519577942" watchObservedRunningTime="2026-06-02 05:40:51.811076234 +0000 UTC m=+903.521525480" Jun 02 05:41:02.802802 ip-10-0-141-22 kubenswrapper[2561]: I0602 05:41:02.802772 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bc6dfc4b6-6jn8w" Jun 02 06:02:12.187015 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:12.186946 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-668ccb6567-ssdv2_66760c9c-b6f0-4f04-b674-a693b716c85b/authorino/0.log" Jun 02 06:02:16.847638 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:16.847605 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7f9b65cdbd-mxjmz_6e9d48c4-d63c-47a9-84be-966dbadca366/nginx/0.log" Jun 02 06:02:17.197893 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:17.197804 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-649d57b67d-4tp6b_bab5d774-5c5d-4bbc-85a9-a2c16961df4b/maas-api/0.log" Jun 02 06:02:17.318782 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:17.318749 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7bc6dfc4b6-6jn8w_bca4a681-0d98-4a1e-86cf-df35d4ed6cc3/manager/0.log" Jun 02 06:02:17.682311 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:17.682285 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f99d6f45-5crx4_7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d/copy-manifests/0.log" Jun 02 06:02:17.693699 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:17.693663 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f99d6f45-5crx4_7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d/manager/0.log" Jun 02 06:02:19.350861 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:19.350825 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-668ccb6567-ssdv2_66760c9c-b6f0-4f04-b674-a693b716c85b/authorino/0.log" Jun 02 06:02:19.699944 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:19.699868 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-55vg7_1057b8a8-6d5d-4775-8b02-53e6e55f1e04/kuadrant-console-plugin/0.log" Jun 02 06:02:20.201166 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:20.201134 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-btkhm_ab3ac6ce-4fcf-4b32-b110-baf01995e4b7/manager/0.log" Jun 02 06:02:20.804290 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:20.804260 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c78f6699c-l4rxp_95e46288-fece-4e36-9b2b-b801570e76ab/kube-auth-proxy/0.log" Jun 02 06:02:21.611328 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:21.611297 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb_111e75a4-b829-4b45-b6b4-970de5fd7c14/storage-initializer/0.log" Jun 02 06:02:21.618475 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:21.618448 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-8krbb_111e75a4-b829-4b45-b6b4-970de5fd7c14/main/0.log" Jun 02 06:02:21.853194 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:21.853161 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh_fefe6d3b-6c83-4031-85b2-fa842ca362e5/storage-initializer/0.log" Jun 02 06:02:21.860895 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:21.860871 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-95bhh_fefe6d3b-6c83-4031-85b2-fa842ca362e5/main/0.log" Jun 02 06:02:21.967558 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:21.967469 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7_87c7e1ee-aaab-433b-b09a-189131fe636c/main/0.log" Jun 02 06:02:21.974741 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:21.974725 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f98tdx7_87c7e1ee-aaab-433b-b09a-189131fe636c/storage-initializer/0.log" Jun 02 06:02:22.089360 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:22.089333 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8_d62f521f-295a-4789-a92b-1435f293bd1f/storage-initializer/0.log" Jun 02 06:02:22.098109 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:22.098083 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-nq8x8_d62f521f-295a-4789-a92b-1435f293bd1f/main/0.log" Jun 02 06:02:29.848118 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:29.848084 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8qmnz_47e6d278-6f80-4c82-857d-564ed5515845/global-pull-secret-syncer/0.log" Jun 02 06:02:29.935173 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:29.935138 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7727j_46aef684-2246-40d2-ae8e-ad3f08626234/konnectivity-agent/0.log" Jun 02 06:02:30.047839 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:30.047810 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-22.ec2.internal_165b8d8bc20dd92d757a5571670d3d63/haproxy/0.log" Jun 02 06:02:33.628491 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:33.628458 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-668ccb6567-ssdv2_66760c9c-b6f0-4f04-b674-a693b716c85b/authorino/0.log" Jun 02 06:02:33.712730 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:33.712704 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-55vg7_1057b8a8-6d5d-4775-8b02-53e6e55f1e04/kuadrant-console-plugin/0.log" Jun 02 06:02:33.920361 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:33.920271 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-btkhm_ab3ac6ce-4fcf-4b32-b110-baf01995e4b7/manager/0.log" Jun 02 06:02:35.558026 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.557991 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-dvbh4_a1021c7e-8fa9-4dac-b2be-17baa7deb980/kube-state-metrics/0.log" Jun 02 06:02:35.580579 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.580552 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-dvbh4_a1021c7e-8fa9-4dac-b2be-17baa7deb980/kube-rbac-proxy-main/0.log" Jun 02 06:02:35.607603 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.607580 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-dvbh4_a1021c7e-8fa9-4dac-b2be-17baa7deb980/kube-rbac-proxy-self/0.log" Jun 02 06:02:35.844156 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.844121 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rnmxr_f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e/node-exporter/0.log" Jun 02 06:02:35.865837 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.865811 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rnmxr_f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e/kube-rbac-proxy/0.log" Jun 02 06:02:35.886689 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.886669 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rnmxr_f60ae33c-91c4-40f5-9c7e-99e43fbc8a7e/init-textfile/0.log" Jun 02 06:02:35.928398 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.928334 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-7cfhl_c841b114-f106-47ab-a637-850293ff966e/kube-rbac-proxy-main/0.log" Jun 02 06:02:35.948667 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.948642 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-7cfhl_c841b114-f106-47ab-a637-850293ff966e/kube-rbac-proxy-self/0.log" Jun 02 06:02:35.968814 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:35.968791 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-7cfhl_c841b114-f106-47ab-a637-850293ff966e/openshift-state-metrics/0.log" Jun 02 06:02:36.012435 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:36.012405 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2d35c990-be4a-4fe8-a01c-8f015ca3e99c/prometheus/0.log" Jun 02 06:02:36.033644 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:36.033621 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2d35c990-be4a-4fe8-a01c-8f015ca3e99c/config-reloader/0.log" Jun 02 06:02:36.056210 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:36.056183 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2d35c990-be4a-4fe8-a01c-8f015ca3e99c/thanos-sidecar/0.log" Jun 02 06:02:36.077489 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:36.077461 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2d35c990-be4a-4fe8-a01c-8f015ca3e99c/kube-rbac-proxy-web/0.log" Jun 02 06:02:36.098721 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:36.098694 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2d35c990-be4a-4fe8-a01c-8f015ca3e99c/kube-rbac-proxy/0.log" Jun 02 06:02:36.126817 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:36.126792 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2d35c990-be4a-4fe8-a01c-8f015ca3e99c/kube-rbac-proxy-thanos/0.log" Jun 02 06:02:36.148777 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:36.148749 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2d35c990-be4a-4fe8-a01c-8f015ca3e99c/init-config-reloader/0.log" Jun 02 06:02:39.075613 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:39.075583 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-555c5c4c65-t7vd7_c45bf84c-5a08-4bb2-a1ef-8f5f1afc9807/console/0.log" Jun 02 06:02:39.108111 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:39.108083 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-cqwx6_8abf55f0-6c9e-4be7-a98b-1f7854ecab74/download-server/0.log" Jun 02 06:02:40.659510 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.659470 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kc7f9_b1df4c97-bd0a-44e3-bc70-1c71520b0eec/dns/0.log" Jun 02 06:02:40.681956 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.681921 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kc7f9_b1df4c97-bd0a-44e3-bc70-1c71520b0eec/kube-rbac-proxy/0.log" Jun 02 06:02:40.810706 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.810676 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5bs54_ecb2d002-0193-47b8-a3a5-c896da8ba212/dns-node-resolver/0.log" Jun 02 06:02:40.825460 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.825426 2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx"] Jun 02 06:02:40.829156 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.829132 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:40.831661 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.831639 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s5gtt\"/\"kube-root-ca.crt\"" Jun 02 06:02:40.831661 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.831654 2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s5gtt\"/\"openshift-service-ca.crt\"" Jun 02 06:02:40.832405 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.832383 2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s5gtt\"/\"default-dockercfg-kd28m\"" Jun 02 06:02:40.841230 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.841204 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx"] Jun 02 06:02:40.924526 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.924431 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-sys\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:40.924526 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.924468 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vslg\" (UniqueName: \"kubernetes.io/projected/cea442d7-9316-489c-a2dc-a5f985db4e74-kube-api-access-7vslg\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:40.924793 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.924546 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-podres\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:40.924793 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.924709 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-lib-modules\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:40.924793 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:40.924780 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-proc\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.025617 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025575 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-sys\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.025617 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025616 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7vslg\" (UniqueName: \"kubernetes.io/projected/cea442d7-9316-489c-a2dc-a5f985db4e74-kube-api-access-7vslg\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.025843 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025694 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-sys\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.025843 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025741 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-podres\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.025843 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025811 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-lib-modules\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.025843 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025834 2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-proc\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.026026 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025854 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-podres\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.026026 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025950 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-proc\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.026026 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.025993 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cea442d7-9316-489c-a2dc-a5f985db4e74-lib-modules\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.033541 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.033518 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vslg\" (UniqueName: \"kubernetes.io/projected/cea442d7-9316-489c-a2dc-a5f985db4e74-kube-api-access-7vslg\") pod \"perf-node-gather-daemonset-xstpx\" (UID: \"cea442d7-9316-489c-a2dc-a5f985db4e74\") " pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.139602 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.139566 2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:41.269527 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.269492 2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx"] Jun 02 06:02:41.272861 ip-10-0-141-22 kubenswrapper[2561]: W0602 06:02:41.272828 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcea442d7_9316_489c_a2dc_a5f985db4e74.slice/crio-24e645df341435879b38b19a0975eaf8086e9d3e2aeb5ba2285e1879714457d2 WatchSource:0}: Error finding container 24e645df341435879b38b19a0975eaf8086e9d3e2aeb5ba2285e1879714457d2: Status 404 returned error can't find the container with id 24e645df341435879b38b19a0975eaf8086e9d3e2aeb5ba2285e1879714457d2 Jun 02 06:02:41.274462 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.274445 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 06:02:41.452524 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.452447 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6658fc4b86-t7pgh_b05a3131-07de-4724-a7bd-e6ee4c6b6dbd/registry/0.log" Jun 02 06:02:41.472310 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:41.472286 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5mg2p_337004e8-eb54-4324-9126-4804d74d0dff/node-ca/0.log" Jun 02 06:02:42.232374 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:42.232340 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" event={"ID":"cea442d7-9316-489c-a2dc-a5f985db4e74","Type":"ContainerStarted","Data":"057b40cb4ef641e7ec07b1c6c86ff1d359666f82a115ee12ba3c7500c60e63b5"} Jun 02 06:02:42.232374 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:42.232376 2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" event={"ID":"cea442d7-9316-489c-a2dc-a5f985db4e74","Type":"ContainerStarted","Data":"24e645df341435879b38b19a0975eaf8086e9d3e2aeb5ba2285e1879714457d2"} Jun 02 06:02:42.232811 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:42.232474 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:42.248081 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:42.248031 2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" podStartSLOduration=2.248017183 podStartE2EDuration="2.248017183s" podCreationTimestamp="2026-06-02 06:02:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 06:02:42.247277099 +0000 UTC m=+2213.957726359" watchObservedRunningTime="2026-06-02 06:02:42.248017183 +0000 UTC m=+2213.958466428" Jun 02 06:02:42.429836 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:42.429811 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c78f6699c-l4rxp_95e46288-fece-4e36-9b2b-b801570e76ab/kube-auth-proxy/0.log" Jun 02 06:02:43.193753 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:43.193721 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rw625_7c31483d-0694-44d8-9125-f41ddf67e3d9/serve-healthcheck-canary/0.log" Jun 02 06:02:43.848832 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:43.848799 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgbc5_7f7780cc-3632-457b-9023-8e60bf763765/kube-rbac-proxy/0.log" Jun 02 06:02:43.867845 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:43.867813 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgbc5_7f7780cc-3632-457b-9023-8e60bf763765/exporter/0.log" Jun 02 06:02:43.887225 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:43.887202 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgbc5_7f7780cc-3632-457b-9023-8e60bf763765/extractor/0.log" Jun 02 06:02:45.877759 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:45.877727 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7f9b65cdbd-mxjmz_6e9d48c4-d63c-47a9-84be-966dbadca366/nginx/0.log" Jun 02 06:02:46.087027 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:46.086995 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-649d57b67d-4tp6b_bab5d774-5c5d-4bbc-85a9-a2c16961df4b/maas-api/0.log" Jun 02 06:02:46.156099 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:46.156016 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7bc6dfc4b6-6jn8w_bca4a681-0d98-4a1e-86cf-df35d4ed6cc3/manager/0.log" Jun 02 06:02:46.267863 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:46.267835 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f99d6f45-5crx4_7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d/manager/0.log" Jun 02 06:02:46.288026 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:46.288005 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f99d6f45-5crx4_7cb4bc5f-d0e1-4a99-ae80-9f773d45d55d/copy-manifests/0.log" Jun 02 06:02:47.794321 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:47.794286 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-647f887c64-pcfzk_b9308d4e-f473-4244-9583-a2a855743679/manager/0.log" Jun 02 06:02:48.246446 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:48.246367 2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s5gtt/perf-node-gather-daemonset-xstpx" Jun 02 06:02:54.487758 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.487732 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vq6j4_420349fd-ebb4-48ed-8c9f-3a60a886d66c/kube-multus-additional-cni-plugins/0.log" Jun 02 06:02:54.507542 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.507517 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vq6j4_420349fd-ebb4-48ed-8c9f-3a60a886d66c/egress-router-binary-copy/0.log" Jun 02 06:02:54.528363 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.528334 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vq6j4_420349fd-ebb4-48ed-8c9f-3a60a886d66c/cni-plugins/0.log" Jun 02 06:02:54.548900 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.548877 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vq6j4_420349fd-ebb4-48ed-8c9f-3a60a886d66c/bond-cni-plugin/0.log" Jun 02 06:02:54.571233 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.571213 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vq6j4_420349fd-ebb4-48ed-8c9f-3a60a886d66c/routeoverride-cni/0.log" Jun 02 06:02:54.590778 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.590758 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vq6j4_420349fd-ebb4-48ed-8c9f-3a60a886d66c/whereabouts-cni-bincopy/0.log" Jun 02 06:02:54.610008 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.609955 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vq6j4_420349fd-ebb4-48ed-8c9f-3a60a886d66c/whereabouts-cni/0.log" Jun 02 06:02:54.703446 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.703419 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-thns8_e7aa2256-b292-46a9-8840-3ac46ae71b59/kube-multus/0.log" Jun 02 06:02:54.721875 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.721843 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4fgvs_a99b8d9f-7723-40bc-be05-b17d605888b4/network-metrics-daemon/0.log" Jun 02 06:02:54.741426 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:54.741368 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4fgvs_a99b8d9f-7723-40bc-be05-b17d605888b4/kube-rbac-proxy/0.log" Jun 02 06:02:55.990262 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:55.990229 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tz8dl_71860b69-2110-4ccf-a8dc-55a86ce967a0/ovn-controller/0.log" Jun 02 06:02:56.018605 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:56.018578 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tz8dl_71860b69-2110-4ccf-a8dc-55a86ce967a0/ovn-acl-logging/0.log" Jun 02 06:02:56.036293 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:56.036268 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tz8dl_71860b69-2110-4ccf-a8dc-55a86ce967a0/kube-rbac-proxy-node/0.log" Jun 02 06:02:56.056479 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:56.056455 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tz8dl_71860b69-2110-4ccf-a8dc-55a86ce967a0/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 06:02:56.075758 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:56.075736 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tz8dl_71860b69-2110-4ccf-a8dc-55a86ce967a0/northd/0.log" Jun 02 06:02:56.097033 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:56.097009 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tz8dl_71860b69-2110-4ccf-a8dc-55a86ce967a0/nbdb/0.log" Jun 02 06:02:56.115777 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:56.115753 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tz8dl_71860b69-2110-4ccf-a8dc-55a86ce967a0/sbdb/0.log" Jun 02 06:02:56.214178 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:56.214134 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tz8dl_71860b69-2110-4ccf-a8dc-55a86ce967a0/ovnkube-controller/0.log" Jun 02 06:02:57.676937 ip-10-0-141-22 kubenswrapper[2561]: I0602 06:02:57.676908 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-9vkts_ddfa33ef-1107-42ae-b2aa-b281becf554b/network-check-target-container/0.log"