Jun 10 17:31:29.914266 ip-10-0-138-237 systemd[1]: Starting Kubernetes Kubelet... Jun 10 17:31:30.301502 ip-10-0-138-237 kubenswrapper[2531]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 17:31:30.301502 ip-10-0-138-237 kubenswrapper[2531]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 10 17:31:30.301502 ip-10-0-138-237 kubenswrapper[2531]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 17:31:30.301502 ip-10-0-138-237 kubenswrapper[2531]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 10 17:31:30.301502 ip-10-0-138-237 kubenswrapper[2531]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 17:31:30.302134 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.301522 2531 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 10 17:31:30.307277 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307261 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 17:31:30.307277 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307276 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307280 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307284 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307287 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307290 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307293 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307296 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307299 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307302 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307306 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307308 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307313 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307316 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307319 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307322 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307326 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307330 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307333 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307336 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 17:31:30.307339 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307338 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307341 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307344 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307347 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307349 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307352 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307355 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307358 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307361 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307363 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307366 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307369 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307371 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307374 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307376 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307379 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307381 2531 feature_gate.go:328] unrecognized feature gate: Example Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307383 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307386 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307388 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 17:31:30.307815 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307391 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307393 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307395 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307398 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307400 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307403 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307405 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307408 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307410 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307413 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307415 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307417 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307420 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307422 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307426 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307428 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307431 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307433 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307436 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307438 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 17:31:30.308576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307441 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307443 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307446 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307449 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307453 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307456 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307458 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307461 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307463 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307465 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307468 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307470 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307473 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307475 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307479 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307482 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307484 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307486 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307489 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307491 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 17:31:30.309091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307494 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307496 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307499 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307501 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307503 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307506 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307510 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307512 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307515 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307517 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307520 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307522 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307525 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307527 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307530 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307532 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307535 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307538 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 17:31:30.309581 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.307540 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 17:31:30.310881 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310865 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 17:31:30.310881 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310880 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310885 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310889 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310891 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310895 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310897 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310900 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310903 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310906 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310908 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310911 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310913 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310916 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310918 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310921 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310923 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310926 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310929 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310931 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 17:31:30.310955 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310948 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310953 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310977 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310981 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310984 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310987 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310990 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310992 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310995 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.310998 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311000 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311002 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311006 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311008 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311012 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311015 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311017 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311020 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311022 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311025 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 17:31:30.311422 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311027 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311029 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311032 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311034 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311037 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311039 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311041 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311044 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311046 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311049 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311051 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311053 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311055 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311058 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311061 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311064 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311066 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311068 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311071 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 17:31:30.311911 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311073 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311076 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311078 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311080 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311083 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311086 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311088 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311091 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311093 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311096 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311099 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311101 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311104 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311106 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311108 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311111 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311113 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311117 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311121 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 17:31:30.312388 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311127 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311130 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311133 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311135 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311138 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311142 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311144 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311147 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311150 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311154 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311156 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311159 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311161 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311164 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311166 2531 feature_gate.go:328] unrecognized feature gate: Example Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311169 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311171 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311174 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311177 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311179 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 17:31:30.312861 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311181 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311545 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311549 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311552 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311555 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311557 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311560 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311563 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311565 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311568 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311570 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311573 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311576 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311579 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311581 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311584 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311586 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311595 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311598 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 17:31:30.313439 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311601 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311604 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311607 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311609 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311612 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311614 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311618 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311622 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311625 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311628 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311630 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311633 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311635 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311642 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311646 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311648 2531 feature_gate.go:328] unrecognized feature gate: Example Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311651 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311654 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311656 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 17:31:30.313915 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311658 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311661 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311663 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311665 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311668 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311670 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311673 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311676 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311678 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311680 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311683 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311686 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311689 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311692 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311694 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311697 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311699 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311702 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311704 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311707 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 17:31:30.314389 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311709 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311711 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311714 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311716 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311719 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311721 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311724 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311727 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311729 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311732 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311734 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311736 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311739 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311741 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311744 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311746 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311748 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311751 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311754 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311756 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 17:31:30.314870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311759 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311761 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311764 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311766 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311768 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311772 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311774 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311776 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311779 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311781 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311784 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311787 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311791 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311794 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311796 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311799 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311801 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311803 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311806 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 17:31:30.315407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311808 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311811 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.311813 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311884 2531 flags.go:64] FLAG: --address="0.0.0.0" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311891 2531 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311897 2531 flags.go:64] FLAG: --anonymous-auth="true" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311901 2531 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311904 2531 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311907 2531 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311911 2531 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311916 2531 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311920 2531 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311923 2531 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311926 2531 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311930 2531 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311947 2531 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311950 2531 flags.go:64] FLAG: --cgroup-root="" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311953 2531 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311956 2531 flags.go:64] FLAG: --client-ca-file="" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311959 2531 flags.go:64] FLAG: --cloud-provider="external" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311963 2531 flags.go:64] FLAG: --cluster-dns="[]" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311967 2531 flags.go:64] FLAG: --cluster-domain="" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311970 2531 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 10 17:31:30.315870 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311973 2531 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311976 2531 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311979 2531 flags.go:64] FLAG: --container-log-max-files="5" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311989 2531 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311992 2531 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311995 2531 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.311998 2531 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312001 2531 flags.go:64] FLAG: --contention-profiling="false" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312004 2531 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312007 2531 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312012 2531 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312015 2531 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312019 2531 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312022 2531 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312025 2531 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312027 2531 flags.go:64] FLAG: --enable-load-reader="false" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312030 2531 flags.go:64] FLAG: --enable-server="true" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312033 2531 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312040 2531 flags.go:64] FLAG: --event-burst="100" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312042 2531 flags.go:64] FLAG: --event-qps="50" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312047 2531 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312049 2531 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312052 2531 flags.go:64] FLAG: --eviction-hard="" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312056 2531 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 10 17:31:30.316475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312059 2531 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312062 2531 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312065 2531 flags.go:64] FLAG: --eviction-soft="" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312067 2531 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312070 2531 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312073 2531 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312076 2531 flags.go:64] FLAG: --experimental-mounter-path="" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312079 2531 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312082 2531 flags.go:64] FLAG: --fail-swap-on="true" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312084 2531 flags.go:64] FLAG: --feature-gates="" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312092 2531 flags.go:64] FLAG: --file-check-frequency="20s" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312095 2531 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312098 2531 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312101 2531 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312104 2531 flags.go:64] FLAG: --healthz-port="10248" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312107 2531 flags.go:64] FLAG: --help="false" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312110 2531 flags.go:64] FLAG: --hostname-override="ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312113 2531 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312117 2531 flags.go:64] FLAG: --http-check-frequency="20s" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312119 2531 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312123 2531 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312126 2531 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312129 2531 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 10 17:31:30.317072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312131 2531 flags.go:64] FLAG: --image-service-endpoint="" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312134 2531 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312137 2531 flags.go:64] FLAG: --kube-api-burst="100" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312139 2531 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312142 2531 flags.go:64] FLAG: --kube-api-qps="50" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312146 2531 flags.go:64] FLAG: --kube-reserved="" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312149 2531 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312152 2531 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312155 2531 flags.go:64] FLAG: --kubelet-cgroups="" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312157 2531 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312160 2531 flags.go:64] FLAG: --lock-file="" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312162 2531 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312165 2531 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312168 2531 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312173 2531 flags.go:64] FLAG: --log-json-split-stream="false" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312177 2531 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312180 2531 flags.go:64] FLAG: --log-text-split-stream="false" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312183 2531 flags.go:64] FLAG: --logging-format="text" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312185 2531 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312189 2531 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312191 2531 flags.go:64] FLAG: --manifest-url="" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312194 2531 flags.go:64] FLAG: --manifest-url-header="" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312198 2531 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312201 2531 flags.go:64] FLAG: --max-open-files="1000000" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312205 2531 flags.go:64] FLAG: --max-pods="110" Jun 10 17:31:30.317656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312207 2531 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312210 2531 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312214 2531 flags.go:64] FLAG: --memory-manager-policy="None" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312217 2531 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312220 2531 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312223 2531 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312226 2531 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312233 2531 flags.go:64] FLAG: --node-status-max-images="50" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312237 2531 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312239 2531 flags.go:64] FLAG: --oom-score-adj="-999" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312242 2531 flags.go:64] FLAG: --pod-cidr="" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312245 2531 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312251 2531 flags.go:64] FLAG: --pod-manifest-path="" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312254 2531 flags.go:64] FLAG: --pod-max-pids="-1" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312257 2531 flags.go:64] FLAG: --pods-per-core="0" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312259 2531 flags.go:64] FLAG: --port="10250" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312262 2531 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312265 2531 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01eabd627f5b97a37" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312268 2531 flags.go:64] FLAG: --qos-reserved="" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312271 2531 flags.go:64] FLAG: --read-only-port="10255" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312274 2531 flags.go:64] FLAG: --register-node="true" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312276 2531 flags.go:64] FLAG: --register-with-taints="" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312284 2531 flags.go:64] FLAG: --registry-burst="10" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312287 2531 flags.go:64] FLAG: --registry-qps="5" Jun 10 17:31:30.318283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312290 2531 flags.go:64] FLAG: --reserved-cpus="" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312292 2531 flags.go:64] FLAG: --reserved-memory="" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312298 2531 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312301 2531 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312304 2531 flags.go:64] FLAG: --rotate-certificates="false" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312307 2531 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312309 2531 flags.go:64] FLAG: --runonce="false" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312312 2531 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312315 2531 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312318 2531 flags.go:64] FLAG: --seccomp-default="false" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312320 2531 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312324 2531 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312327 2531 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312330 2531 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312332 2531 flags.go:64] FLAG: --storage-driver-password="root" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312335 2531 flags.go:64] FLAG: --storage-driver-secure="false" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312338 2531 flags.go:64] FLAG: --storage-driver-table="stats" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312341 2531 flags.go:64] FLAG: --storage-driver-user="root" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312343 2531 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312346 2531 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312350 2531 flags.go:64] FLAG: --system-cgroups="" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312353 2531 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312358 2531 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312361 2531 flags.go:64] FLAG: --tls-cert-file="" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312363 2531 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 10 17:31:30.318886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312367 2531 flags.go:64] FLAG: --tls-min-version="" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312369 2531 flags.go:64] FLAG: --tls-private-key-file="" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312372 2531 flags.go:64] FLAG: --topology-manager-policy="none" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312374 2531 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312377 2531 flags.go:64] FLAG: --topology-manager-scope="container" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312380 2531 flags.go:64] FLAG: --v="2" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312384 2531 flags.go:64] FLAG: --version="false" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312388 2531 flags.go:64] FLAG: --vmodule="" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312392 2531 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.312395 2531 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312505 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312509 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312512 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312515 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312518 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312521 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312524 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312527 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312531 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312534 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312536 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312539 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 17:31:30.319499 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312542 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312544 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312547 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312550 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312552 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312556 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312559 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312561 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312564 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312566 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312569 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312572 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312574 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312577 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312579 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312582 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312584 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312587 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312589 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312593 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 17:31:30.320034 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312597 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312599 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312602 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312604 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312607 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312610 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312613 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312615 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312619 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312622 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312625 2531 feature_gate.go:328] unrecognized feature gate: Example Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312629 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312632 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312635 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312638 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312641 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312644 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312647 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312650 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 17:31:30.320536 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312652 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312655 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312658 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312660 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312663 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312665 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312668 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312670 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312673 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312675 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312678 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312680 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312683 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312685 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312688 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312691 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312693 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312695 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312698 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312700 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 17:31:30.321114 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312703 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312706 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312709 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312720 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312722 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312725 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312727 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312730 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312732 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312735 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312738 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312741 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312744 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312746 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312748 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312752 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312756 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312759 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312761 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 17:31:30.321596 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312763 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312766 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312768 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312771 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312773 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312776 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312779 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312782 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.312784 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.313521 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.320825 2531 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.320839 2531 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.321584 2531 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.321592 2531 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321648 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 17:31:30.322069 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321654 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321658 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321661 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321664 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321667 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321670 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321673 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321675 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321678 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321681 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321683 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321686 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321688 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321691 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321693 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321695 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321698 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321701 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321703 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321706 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 17:31:30.322438 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321708 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321711 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321713 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321716 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321719 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321723 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321727 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321730 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321733 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321736 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321739 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321741 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321744 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321746 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321749 2531 feature_gate.go:328] unrecognized feature gate: Example Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321751 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321754 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321757 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321759 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321762 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 17:31:30.322914 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321765 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321767 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321769 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321772 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321774 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321777 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321779 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321783 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321787 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321790 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321792 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321794 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321797 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321801 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321804 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321807 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321811 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321813 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321816 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 17:31:30.323426 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321819 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321821 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321824 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321827 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321829 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321832 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321834 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321836 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321839 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321841 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321844 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321846 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321848 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321851 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321853 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321856 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321858 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321860 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321863 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321865 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 17:31:30.323880 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321868 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321870 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321872 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321875 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321877 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321879 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321882 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321884 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321887 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321890 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321893 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321896 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321899 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321901 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321903 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321906 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321908 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321911 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 17:31:30.324364 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.321913 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.321918 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322043 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322048 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322051 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322053 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322056 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322058 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322061 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322063 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322065 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322069 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322073 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322076 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322079 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 17:31:30.324829 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322081 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322084 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322087 2531 feature_gate.go:328] unrecognized feature gate: Example Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322089 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322091 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322094 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322096 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322099 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322101 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322104 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322106 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322109 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322112 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322114 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322116 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322119 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322121 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322123 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322126 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322128 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 17:31:30.325217 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322130 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322133 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322135 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322138 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322140 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322142 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322145 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322147 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322150 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322152 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322154 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322157 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322159 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322161 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322164 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322166 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322169 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322171 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322174 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 17:31:30.325700 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322176 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322179 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322181 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322183 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322186 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322189 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322191 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322193 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322196 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322198 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322201 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322203 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322205 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322208 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322210 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322213 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322215 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322217 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322220 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322222 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 17:31:30.326244 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322224 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322227 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322229 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322232 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322235 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322237 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322240 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322242 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322245 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322247 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322249 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322252 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322255 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322258 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322261 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322264 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322266 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322269 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322272 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322274 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 17:31:30.326727 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322277 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322280 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322284 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322287 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322289 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322292 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:30.322295 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.322300 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.322393 2531 server.go:962] "Client rotation is on, will bootstrap in background" Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.324799 2531 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.326367 2531 server.go:1019] "Starting client certificate rotation" Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.326471 2531 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 17:31:30.327216 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.326506 2531 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 17:31:30.328878 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.328865 2531 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 17:31:30.337251 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.337229 2531 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 17:31:30.347309 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.347289 2531 log.go:25] "Validated CRI v1 runtime API" Jun 10 17:31:30.350444 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.350420 2531 log.go:25] "Validated CRI v1 image API" Jun 10 17:31:30.352142 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.352126 2531 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 10 17:31:30.354511 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.354491 2531 fs.go:135] Filesystem UUIDs: map[71919240-02f8-4601-9cde-70f5fc4542ce:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cdf4cbbc-c5ed-4aba-a48d-9c7d968b9e99:/dev/nvme0n1p3] Jun 10 17:31:30.354582 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.354511 2531 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 10 17:31:30.359402 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.359376 2531 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 17:31:30.359626 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.359515 2531 manager.go:216] Machine: {Timestamp:2026-06-10 17:31:30.358176792 +0000 UTC m=+0.326382821 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094858 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2370ee2c3742c4733753e13cce9ff6 SystemUUID:ec2370ee-2c37-42c4-7337-53e13cce9ff6 BootID:7c057ddc-14d5-42a1-b402-98da112e62de Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:89:f1:4a:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:89:f1:4a:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:22:44:ab:d2:13 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 10 17:31:30.359626 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.359621 2531 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 10 17:31:30.359774 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.359726 2531 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 10 17:31:30.362854 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.362826 2531 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 10 17:31:30.363024 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.362856 2531 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-237.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 10 17:31:30.364273 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.364261 2531 topology_manager.go:138] "Creating topology manager with none policy" Jun 10 17:31:30.364335 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.364279 2531 container_manager_linux.go:309] "Creating device plugin manager" Jun 10 17:31:30.364335 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.364289 2531 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 17:31:30.365024 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.365012 2531 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 17:31:30.366333 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.366321 2531 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 10 17:31:30.369901 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.369887 2531 state_mem.go:36] "Initialized new in-memory state store" Jun 10 17:31:30.370046 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.370034 2531 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 10 17:31:30.370107 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.370075 2531 kubelet.go:486] "Attempting to sync node with API server" Jun 10 17:31:30.370107 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.370089 2531 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 10 17:31:30.370190 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.370115 2531 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 10 17:31:30.370190 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.370130 2531 kubelet.go:398] "Adding apiserver pod source" Jun 10 17:31:30.370190 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.370142 2531 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 10 17:31:30.370512 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.370498 2531 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 17:31:30.370579 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.370521 2531 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 17:31:30.372861 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.372846 2531 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 10 17:31:30.374890 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.374877 2531 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 10 17:31:30.374949 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.374907 2531 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376291 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376307 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376314 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376320 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376330 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376335 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376344 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376349 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376356 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376362 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376379 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 10 17:31:30.376931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.376385 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 10 17:31:30.377276 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.377223 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 10 17:31:30.377276 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.377233 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 10 17:31:30.380420 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.380401 2531 server.go:1268] "Started kubelet" Jun 10 17:31:30.380631 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.380575 2531 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 10 17:31:30.380774 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.380760 2531 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 10 17:31:30.380953 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.380612 2531 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 10 17:31:30.381206 ip-10-0-138-237 systemd[1]: Started Kubernetes Kubelet. Jun 10 17:31:30.383468 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.383448 2531 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 10 17:31:30.383649 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.383629 2531 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5gmqf" Jun 10 17:31:30.384212 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.384188 2531 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 10 17:31:30.384313 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.384288 2531 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 10 17:31:30.384458 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.384435 2531 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 10 17:31:30.386165 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.386147 2531 server.go:310] "Adding debug handlers to kubelet server" Jun 10 17:31:30.387254 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.387234 2531 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5gmqf" Jun 10 17:31:30.389747 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.389717 2531 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 10 17:31:30.391735 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.390972 2531 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-237.ec2.internal.18b7c88e1f0ff801 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-237.ec2.internal,UID:ip-10-0-138-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-237.ec2.internal,},FirstTimestamp:2026-06-10 17:31:30.380371969 +0000 UTC m=+0.348578000,LastTimestamp:2026-06-10 17:31:30.380371969 +0000 UTC m=+0.348578000,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-237.ec2.internal,}" Jun 10 17:31:30.394260 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.394245 2531 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 10 17:31:30.394319 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.394259 2531 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 10 17:31:30.394367 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.394315 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:30.394995 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.394975 2531 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 10 17:31:30.394995 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.394978 2531 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 10 17:31:30.395145 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395006 2531 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 10 17:31:30.395145 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395121 2531 factory.go:55] Registering systemd factory Jun 10 17:31:30.395237 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395155 2531 factory.go:223] Registration of the systemd container factory successfully Jun 10 17:31:30.395237 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395169 2531 reconstruct.go:97] "Volume reconstruction finished" Jun 10 17:31:30.395237 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395177 2531 reconciler.go:29] "Reconciler: start to sync state" Jun 10 17:31:30.395361 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395323 2531 factory.go:153] Registering CRI-O factory Jun 10 17:31:30.395361 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395333 2531 factory.go:223] Registration of the crio container factory successfully Jun 10 17:31:30.395458 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395376 2531 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 10 17:31:30.395458 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395393 2531 factory.go:103] Registering Raw factory Jun 10 17:31:30.395458 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395403 2531 manager.go:1196] Started watching for new ooms in manager Jun 10 17:31:30.395805 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.395786 2531 manager.go:349] Starting recovery of all containers Jun 10 17:31:30.397658 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.397641 2531 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 17:31:30.401499 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.401480 2531 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-237.ec2.internal\" not found" node="ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.405656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.405639 2531 manager.go:354] Recovery completed Jun 10 17:31:30.409728 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.409717 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 17:31:30.411851 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.411835 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory" Jun 10 17:31:30.411903 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.411869 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 17:31:30.411903 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.411885 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID" Jun 10 17:31:30.412369 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.412359 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 17:31:30.412450 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.412360 2531 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 10 17:31:30.412481 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.412450 2531 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 10 17:31:30.412481 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.412466 2531 state_mem.go:36] "Initialized new in-memory state store" Jun 10 17:31:30.413446 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.413431 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory" Jun 10 17:31:30.413518 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.413457 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 17:31:30.413518 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.413467 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID" Jun 10 17:31:30.413709 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.413698 2531 policy_none.go:49] "None policy: Start" Jun 10 17:31:30.413745 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.413712 2531 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 10 17:31:30.413745 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.413722 2531 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 10 17:31:30.415084 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.415074 2531 policy_none.go:47] "Start" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.453533 2531 manager.go:337] "Starting Device Plugin manager" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.453561 2531 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.453571 2531 server.go:87] "Starting device plugin registration server" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.453752 2531 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.453764 2531 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.453850 2531 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.453930 2531 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.453959 2531 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.454407 2531 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.454465 2531 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.462031 2531 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.463321 2531 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.463354 2531 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.463371 2531 kubelet.go:2448] "Starting kubelet main sync loop" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.463428 2531 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 10 17:31:30.465786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.465508 2531 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 17:31:30.554804 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.554763 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 17:31:30.555651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.555634 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory" Jun 10 17:31:30.555747 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.555665 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 17:31:30.555747 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.555681 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID" Jun 10 17:31:30.555747 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.555704 2531 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.564141 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.564122 2531 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.564239 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.564148 2531 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-237.ec2.internal\": node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:30.564308 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.564293 2531 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"] Jun 10 17:31:30.564360 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.564351 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 17:31:30.565101 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.565087 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory" Jun 10 17:31:30.565186 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.565111 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 17:31:30.565186 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.565121 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID" Jun 10 17:31:30.566447 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.566435 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 17:31:30.566610 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.566594 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.566680 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.566663 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 17:31:30.567131 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.567112 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory" Jun 10 17:31:30.567131 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.567134 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 17:31:30.567288 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.567143 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID" Jun 10 17:31:30.567288 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.567159 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory" Jun 10 17:31:30.567288 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.567176 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 17:31:30.567288 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.567185 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID" Jun 10 17:31:30.568499 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.568483 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.568577 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.568509 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 17:31:30.569127 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.569112 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory" Jun 10 17:31:30.569211 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.569142 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 17:31:30.569211 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.569156 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID" Jun 10 17:31:30.576791 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.576750 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:30.591533 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.591514 2531 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-237.ec2.internal\" not found" node="ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.594631 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.594617 2531 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-237.ec2.internal\" not found" node="ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.677658 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.677636 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:30.696090 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.696062 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281b45fad3c7c003f8a039bb575ccb10-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"281b45fad3c7c003f8a039bb575ccb10\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.696090 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.696088 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281b45fad3c7c003f8a039bb575ccb10-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"281b45fad3c7c003f8a039bb575ccb10\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.696220 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.696107 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/193fc2f766e2d337515229dea4784854-config\") pod \"kube-apiserver-proxy-ip-10-0-138-237.ec2.internal\" (UID: \"193fc2f766e2d337515229dea4784854\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.778545 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.778519 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:30.796877 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.796855 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281b45fad3c7c003f8a039bb575ccb10-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"281b45fad3c7c003f8a039bb575ccb10\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.796949 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.796924 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281b45fad3c7c003f8a039bb575ccb10-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"281b45fad3c7c003f8a039bb575ccb10\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.796998 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.796881 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281b45fad3c7c003f8a039bb575ccb10-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"281b45fad3c7c003f8a039bb575ccb10\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.796998 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.796966 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281b45fad3c7c003f8a039bb575ccb10-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"281b45fad3c7c003f8a039bb575ccb10\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.797072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.797000 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/193fc2f766e2d337515229dea4784854-config\") pod \"kube-apiserver-proxy-ip-10-0-138-237.ec2.internal\" (UID: \"193fc2f766e2d337515229dea4784854\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.797072 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.797036 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/193fc2f766e2d337515229dea4784854-config\") pod \"kube-apiserver-proxy-ip-10-0-138-237.ec2.internal\" (UID: \"193fc2f766e2d337515229dea4784854\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.879327 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.879264 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:30.896541 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.896521 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.900369 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:30.900249 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" Jun 10 17:31:30.979579 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:30.979543 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:31.080099 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.080069 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:31.180610 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.180541 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:31.281281 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.281133 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found" Jun 10 17:31:31.313121 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.313102 2531 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 17:31:31.325216 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.325192 2531 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 10 17:31:31.325334 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.325316 2531 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 10 17:31:31.325383 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.325355 2531 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 10 17:31:31.325426 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.325362 2531 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 10 17:31:31.370517 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.370491 2531 apiserver.go:52] "Watching apiserver" Jun 10 17:31:31.384303 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.384277 2531 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 17:31:31.387125 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.387106 2531 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 10 17:31:31.388174 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.388152 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-rk5fq","openshift-dns/node-resolver-n8cmd","openshift-image-registry/node-ca-vsqlg","openshift-multus/multus-additional-cni-plugins-2zd59","openshift-multus/multus-hskvc","openshift-network-diagnostics/network-check-target-8l6dz","openshift-network-operator/iptables-alerter-bv2hz","kube-system/global-pull-secret-syncer-6wslj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7","openshift-cluster-node-tuning-operator/tuned-z5xvj","openshift-multus/network-metrics-daemon-f5f66","openshift-ovn-kubernetes/ovnkube-node-k9jjg"] Jun 10 17:31:31.389589 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.389564 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:31.390949 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.390922 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.391538 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.391523 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 10 17:31:31.391689 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.391659 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-58zvf\"" Jun 10 17:31:31.391689 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.391672 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 10 17:31:31.392141 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.392123 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.392469 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.392443 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod193fc2f766e2d337515229dea4784854.slice/crio-3a92f0ed84fb4ffe1881705e3a25f3998fc248bbc7d3872e096c360c66991cbc WatchSource:0}: Error finding container 3a92f0ed84fb4ffe1881705e3a25f3998fc248bbc7d3872e096c360c66991cbc: Status 404 returned error can't find the container with id 3a92f0ed84fb4ffe1881705e3a25f3998fc248bbc7d3872e096c360c66991cbc Jun 10 17:31:31.392824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.392807 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 10 17:31:31.392923 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.392820 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 10 17:31:31.392923 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.392777 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xjqnq\"" Jun 10 17:31:31.393448 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.393423 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.393650 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.393633 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 10 17:31:31.393708 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.393698 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 10 17:31:31.393930 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.393914 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 10 17:31:31.394129 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.394077 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-09 17:26:30 +0000 UTC" deadline="2028-03-20 12:17:06.2550264 +0000 UTC" Jun 10 17:31:31.394173 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.394132 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15570h45m34.860898861s" Jun 10 17:31:31.394173 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.394147 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nxg9h\"" Jun 10 17:31:31.394326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.394311 2531 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 10 17:31:31.394428 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.394414 2531 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" Jun 10 17:31:31.394648 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.394634 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.395958 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.395374 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 10 17:31:31.395958 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.395487 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2ld9h\"" Jun 10 17:31:31.395958 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.395676 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 10 17:31:31.395958 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.395716 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 10 17:31:31.395958 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.395874 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 10 17:31:31.396223 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.396193 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 10 17:31:31.396336 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.396276 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:31.396386 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.396336 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:31.396573 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.396552 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 10 17:31:31.396573 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.396565 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mnv8w\"" Jun 10 17:31:31.397523 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.397381 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.398522 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.398509 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399016 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.399167 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399203 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399260 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399546 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399622 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/111e361e-8521-49d3-9685-9b21257ea3d3-serviceca\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399650 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399670 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k9b8j\"" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399676 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:31.399730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399733 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399761 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-cni-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399784 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02074e9f-b06c-412e-beee-54138e398848-cni-binary-copy\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399822 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-k8s-cni-cncf-io\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399854 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-cni-bin\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399883 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-kubelet\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399908 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02074e9f-b06c-412e-beee-54138e398848-multus-daemon-config\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399949 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04b03051-fc01-4dac-a94b-b7ddc921fdf2-hosts-file\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399972 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fx5m\" (UniqueName: \"kubernetes.io/projected/04b03051-fc01-4dac-a94b-b7ddc921fdf2-kube-api-access-9fx5m\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.399994 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/111e361e-8521-49d3-9685-9b21257ea3d3-host\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400023 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-os-release\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400057 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bc8234a-907c-41cd-9900-f4183678e844-konnectivity-ca\") pod \"konnectivity-agent-rk5fq\" (UID: \"6bc8234a-907c-41cd-9900-f4183678e844\") " pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400078 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cnibin\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400124 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kghh\" (UniqueName: \"kubernetes.io/projected/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-kube-api-access-2kghh\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400154 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-system-cni-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400178 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-cni-multus\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.400285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400201 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-system-cni-dir\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400224 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400246 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-os-release\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400270 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-netns\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400293 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-conf-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400319 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bc8234a-907c-41cd-9900-f4183678e844-agent-certs\") pod \"konnectivity-agent-rk5fq\" (UID: \"6bc8234a-907c-41cd-9900-f4183678e844\") " pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400352 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-multus-certs\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400383 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400432 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400454 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-hostroot\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400479 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-etc-kubernetes\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400512 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5rrp\" (UniqueName: \"kubernetes.io/projected/02074e9f-b06c-412e-beee-54138e398848-kube-api-access-c5rrp\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400551 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b03051-fc01-4dac-a94b-b7ddc921fdf2-tmp-dir\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400590 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwr2p\" (UniqueName: \"kubernetes.io/projected/111e361e-8521-49d3-9685-9b21257ea3d3-kube-api-access-bwr2p\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400613 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-cnibin\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.400653 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-socket-dir-parent\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.401931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.401909 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.402602 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.402577 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 10 17:31:31.402826 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.402809 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 10 17:31:31.403046 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.403011 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 10 17:31:31.405376 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.403200 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sj5vj\"" Jun 10 17:31:31.405376 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.403986 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wglp2\"" Jun 10 17:31:31.405376 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.404126 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 10 17:31:31.405376 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.404486 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 10 17:31:31.405376 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.404755 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:31.405376 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.404854 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:31.406170 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.406152 2531 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 17:31:31.406426 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.406406 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.406426 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.406418 2531 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" Jun 10 17:31:31.406734 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.406717 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"] Jun 10 17:31:31.408269 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.408250 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 10 17:31:31.408359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.408275 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 10 17:31:31.408359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.408283 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fxfcs\"" Jun 10 17:31:31.408546 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.408532 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 10 17:31:31.408634 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.408620 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 10 17:31:31.408690 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.408670 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 10 17:31:31.408856 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.408848 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 10 17:31:31.409385 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.409370 2531 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 17:31:31.417151 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.417132 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281b45fad3c7c003f8a039bb575ccb10.slice/crio-0f5843719f2d59a1648b5845738ce0e2311803aa92c9d648676cbc9363bfe205 WatchSource:0}: Error finding container 0f5843719f2d59a1648b5845738ce0e2311803aa92c9d648676cbc9363bfe205: Status 404 returned error can't find the container with id 0f5843719f2d59a1648b5845738ce0e2311803aa92c9d648676cbc9363bfe205 Jun 10 17:31:31.417699 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.417679 2531 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 17:31:31.417781 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.417741 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"] Jun 10 17:31:31.437171 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.437123 2531 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pr588" Jun 10 17:31:31.444633 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.444617 2531 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pr588" Jun 10 17:31:31.467070 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.467033 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" event={"ID":"281b45fad3c7c003f8a039bb575ccb10","Type":"ContainerStarted","Data":"0f5843719f2d59a1648b5845738ce0e2311803aa92c9d648676cbc9363bfe205"} Jun 10 17:31:31.467930 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.467912 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" event={"ID":"193fc2f766e2d337515229dea4784854","Type":"ContainerStarted","Data":"3a92f0ed84fb4ffe1881705e3a25f3998fc248bbc7d3872e096c360c66991cbc"} Jun 10 17:31:31.496005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.495987 2531 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 10 17:31:31.501702 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.501678 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b03051-fc01-4dac-a94b-b7ddc921fdf2-tmp-dir\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.501775 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.501703 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-cnibin\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.502048 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.501720 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-kubelet-config\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.502153 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502130 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.502204 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502149 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-cnibin\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.502249 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502213 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-systemd-units\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.502341 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502255 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/111e361e-8521-49d3-9685-9b21257ea3d3-serviceca\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.502341 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502264 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b03051-fc01-4dac-a94b-b7ddc921fdf2-tmp-dir\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.502341 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502324 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.502472 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502359 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:31.502472 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502395 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-sys-fs\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.502599 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502548 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fj9k\" (UniqueName: \"kubernetes.io/projected/49901ffb-82bb-4c42-a126-2eef1b1c2b17-kube-api-access-6fj9k\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.502599 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502571 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.502758 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502737 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/111e361e-8521-49d3-9685-9b21257ea3d3-serviceca\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.502867 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502839 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-kubernetes\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.502964 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502896 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-slash\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.503019 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502956 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.503019 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.502993 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-k8s-cni-cncf-io\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.503110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503032 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-cni-bin\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.503110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503041 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-k8s-cni-cncf-io\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.503110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503073 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-kubelet\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.503110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503094 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-cni-bin\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.503110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503104 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02074e9f-b06c-412e-beee-54138e398848-multus-daemon-config\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.503326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503138 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-socket-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.503326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503141 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-kubelet\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.503326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503170 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-etc-selinux\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.503326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503195 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-var-lib-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.503326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503278 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-etc-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503322 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04b03051-fc01-4dac-a94b-b7ddc921fdf2-hosts-file\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503365 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c6b3a40-5850-42d2-b135-fb0bf1820689-tmp\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503397 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q6bw\" (UniqueName: \"kubernetes.io/projected/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-kube-api-access-5q6bw\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503421 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bc8234a-907c-41cd-9900-f4183678e844-konnectivity-ca\") pod \"konnectivity-agent-rk5fq\" (UID: \"6bc8234a-907c-41cd-9900-f4183678e844\") " pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503450 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cnibin\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503480 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-cni-multus\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503525 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-device-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503553 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysconfig\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.503600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503579 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.504005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503606 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-os-release\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503639 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02074e9f-b06c-412e-beee-54138e398848-multus-daemon-config\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503649 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-netns\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503691 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-run\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.504005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503720 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-env-overrides\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.504005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503771 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04b03051-fc01-4dac-a94b-b7ddc921fdf2-hosts-file\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.504005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503929 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovnkube-script-lib\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.504005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.503984 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-multus-certs\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504015 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/155270d8-7663-4a7e-bc21-0af6009f4303-iptables-alerter-script\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504045 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-systemd\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504104 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovn-node-metrics-cert\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504128 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-hostroot\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504156 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-etc-kubernetes\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504183 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-modprobe-d\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504225 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-tuned\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504268 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-run-netns\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504291 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-cni-bin\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504311 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.504344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504335 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwr2p\" (UniqueName: \"kubernetes.io/projected/111e361e-8521-49d3-9685-9b21257ea3d3-kube-api-access-bwr2p\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.504819 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504405 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cnibin\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.504819 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504684 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-os-release\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504819 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504773 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-netns\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504967 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504857 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bc8234a-907c-41cd-9900-f4183678e844-konnectivity-ca\") pod \"konnectivity-agent-rk5fq\" (UID: \"6bc8234a-907c-41cd-9900-f4183678e844\") " pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:31.504967 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504875 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-run-multus-certs\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.504967 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504932 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-host-var-lib-cni-multus\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.505148 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.504980 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-hostroot\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.505148 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505136 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-etc-kubernetes\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.505233 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505164 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-socket-dir-parent\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.505276 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505218 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.505276 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505253 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-socket-dir-parent\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.505365 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505291 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-systemd\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.505365 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505343 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-log-socket\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.505471 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505457 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-var-lib-kubelet\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.505662 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505482 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxzzt\" (UniqueName: \"kubernetes.io/projected/9c6b3a40-5850-42d2-b135-fb0bf1820689-kube-api-access-vxzzt\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.505754 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505669 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:31.505754 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505723 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-cni-netd\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.505823 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.505762 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.506268 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506241 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-cni-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.506311 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506280 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-cni-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.506311 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506287 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02074e9f-b06c-412e-beee-54138e398848-cni-binary-copy\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.506371 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506330 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-registration-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.506371 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506357 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-lib-modules\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.506428 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506381 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-ovn\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.506428 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506408 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fx5m\" (UniqueName: \"kubernetes.io/projected/04b03051-fc01-4dac-a94b-b7ddc921fdf2-kube-api-access-9fx5m\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.506528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506434 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/111e361e-8521-49d3-9685-9b21257ea3d3-host\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.506528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506457 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-os-release\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.506528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506478 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-host\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.506528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506502 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kghh\" (UniqueName: \"kubernetes.io/projected/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-kube-api-access-2kghh\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.506528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506500 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/111e361e-8521-49d3-9685-9b21257ea3d3-host\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.506740 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506526 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-system-cni-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.506740 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506567 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-os-release\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.506740 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506637 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/155270d8-7663-4a7e-bc21-0af6009f4303-host-slash\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.506740 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506660 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.506740 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506675 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-system-cni-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.506740 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506726 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-kubelet\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506748 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-node-log\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506757 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02074e9f-b06c-412e-beee-54138e398848-cni-binary-copy\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506762 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovnkube-config\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506787 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-system-cni-dir\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506807 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-conf-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506823 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-dbus\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506838 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysctl-conf\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506851 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-sys\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506820 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-system-cni-dir\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506876 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02074e9f-b06c-412e-beee-54138e398848-multus-conf-dir\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506900 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hct55\" (UniqueName: \"kubernetes.io/projected/d9f00b72-1079-4ead-a63f-5761b40d3166-kube-api-access-hct55\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:31.506932 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506919 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bc8234a-907c-41cd-9900-f4183678e844-agent-certs\") pod \"konnectivity-agent-rk5fq\" (UID: \"6bc8234a-907c-41cd-9900-f4183678e844\") " pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:31.507359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506965 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bdc8\" (UniqueName: \"kubernetes.io/projected/155270d8-7663-4a7e-bc21-0af6009f4303-kube-api-access-5bdc8\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.507359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506981 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.507359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.506997 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-run-ovn-kubernetes\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.507359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.507012 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.507359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.507039 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5rrp\" (UniqueName: \"kubernetes.io/projected/02074e9f-b06c-412e-beee-54138e398848-kube-api-access-c5rrp\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.507359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.507059 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysctl-d\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.507359 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.507284 2531 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 10 17:31:31.507576 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.507401 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 17:31:31.507576 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.507416 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 17:31:31.507576 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.507418 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.507576 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.507425 2531 projected.go:196] Error preparing data for projected volume kube-api-access-s7zq5 for pod openshift-network-diagnostics/network-check-target-8l6dz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:31.507576 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.507524 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5 podName:b15fa61d-010f-4536-9329-1b5b95efc2d4 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:32.007502765 +0000 UTC m=+1.975708797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s7zq5" (UniqueName: "kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5") pod "network-check-target-8l6dz" (UID: "b15fa61d-010f-4536-9329-1b5b95efc2d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:31.510398 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.510357 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bc8234a-907c-41cd-9900-f4183678e844-agent-certs\") pod \"konnectivity-agent-rk5fq\" (UID: \"6bc8234a-907c-41cd-9900-f4183678e844\") " pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:31.511924 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.511906 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwr2p\" (UniqueName: \"kubernetes.io/projected/111e361e-8521-49d3-9685-9b21257ea3d3-kube-api-access-bwr2p\") pod \"node-ca-vsqlg\" (UID: \"111e361e-8521-49d3-9685-9b21257ea3d3\") " pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.513700 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.513683 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kghh\" (UniqueName: \"kubernetes.io/projected/f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8-kube-api-access-2kghh\") pod \"multus-additional-cni-plugins-2zd59\" (UID: \"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8\") " pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.517148 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.517130 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5rrp\" (UniqueName: \"kubernetes.io/projected/02074e9f-b06c-412e-beee-54138e398848-kube-api-access-c5rrp\") pod \"multus-hskvc\" (UID: \"02074e9f-b06c-412e-beee-54138e398848\") " pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.519304 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.519288 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fx5m\" (UniqueName: \"kubernetes.io/projected/04b03051-fc01-4dac-a94b-b7ddc921fdf2-kube-api-access-9fx5m\") pod \"node-resolver-n8cmd\" (UID: \"04b03051-fc01-4dac-a94b-b7ddc921fdf2\") " pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.607578 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607548 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-slash\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.607578 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607579 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607598 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-socket-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607618 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-etc-selinux\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607633 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-var-lib-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607648 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-etc-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607654 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-slash\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607666 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607664 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c6b3a40-5850-42d2-b135-fb0bf1820689-tmp\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607710 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-etc-selinux\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607732 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-etc-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607717 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q6bw\" (UniqueName: \"kubernetes.io/projected/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-kube-api-access-5q6bw\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.607760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607755 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-var-lib-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607774 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-device-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607798 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysconfig\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607814 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-device-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607822 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-run\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607844 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysconfig\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607847 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-env-overrides\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607872 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovnkube-script-lib\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607879 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-run\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607891 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-socket-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607912 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/155270d8-7663-4a7e-bc21-0af6009f4303-iptables-alerter-script\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607966 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-systemd\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.607986 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovn-node-metrics-cert\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608009 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-modprobe-d\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608030 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-tuned\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608051 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-run-netns\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608073 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-cni-bin\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.608275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608096 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608124 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-systemd\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608130 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-modprobe-d\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608148 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-log-socket\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608086 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-systemd\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608174 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-var-lib-kubelet\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608178 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-run-netns\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608229 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608229 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-log-socket\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608260 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-systemd\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608292 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxzzt\" (UniqueName: \"kubernetes.io/projected/9c6b3a40-5850-42d2-b135-fb0bf1820689-kube-api-access-vxzzt\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608284 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-var-lib-kubelet\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608326 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608351 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-cni-netd\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608325 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-cni-bin\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608363 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-env-overrides\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608390 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-cni-netd\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.609051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608394 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-registration-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608444 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-lib-modules\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608454 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/155270d8-7663-4a7e-bc21-0af6009f4303-iptables-alerter-script\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.608464 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608470 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-ovn\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608492 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-registration-dir\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608546 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-ovn\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608560 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-host\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608565 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-lib-modules\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608589 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/155270d8-7663-4a7e-bc21-0af6009f4303-host-slash\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608596 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-host\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608612 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-kubelet\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608640 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-kubelet\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.608650 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:32.108610175 +0000 UTC m=+2.076816193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608651 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/155270d8-7663-4a7e-bc21-0af6009f4303-host-slash\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608696 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-node-log\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608712 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovnkube-script-lib\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608736 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovnkube-config\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608758 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-node-log\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608763 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-dbus\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608789 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysctl-conf\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608814 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-sys\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608841 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hct55\" (UniqueName: \"kubernetes.io/projected/d9f00b72-1079-4ead-a63f-5761b40d3166-kube-api-access-hct55\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608842 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-dbus\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608865 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-sys\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608878 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bdc8\" (UniqueName: \"kubernetes.io/projected/155270d8-7663-4a7e-bc21-0af6009f4303-kube-api-access-5bdc8\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608912 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608954 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysctl-conf\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.608987 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-run-ovn-kubernetes\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609010 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-run-openvswitch\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609017 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysctl-d\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609059 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-host-run-ovn-kubernetes\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609062 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-kubelet-config\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609103 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.610918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609109 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-kubelet-config\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609160 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-systemd-units\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609133 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-systemd-units\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609247 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovnkube-config\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609293 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-sysctl-d\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.609226 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609342 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-sys-fs\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:31.609371 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret podName:70f889fb-10f8-4bc7-a0a2-42e0ab79dc87 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:32.10935402 +0000 UTC m=+2.077560037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret") pod "global-pull-secret-syncer-6wslj" (UID: "70f889fb-10f8-4bc7-a0a2-42e0ab79dc87") : object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609395 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49901ffb-82bb-4c42-a126-2eef1b1c2b17-sys-fs\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609405 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fj9k\" (UniqueName: \"kubernetes.io/projected/49901ffb-82bb-4c42-a126-2eef1b1c2b17-kube-api-access-6fj9k\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609432 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-kubernetes\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.609502 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-kubernetes\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.610608 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c6b3a40-5850-42d2-b135-fb0bf1820689-tmp\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.610747 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c6b3a40-5850-42d2-b135-fb0bf1820689-etc-tuned\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.611425 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.611179 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-ovn-node-metrics-cert\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.616391 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.616346 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxzzt\" (UniqueName: \"kubernetes.io/projected/9c6b3a40-5850-42d2-b135-fb0bf1820689-kube-api-access-vxzzt\") pod \"tuned-z5xvj\" (UID: \"9c6b3a40-5850-42d2-b135-fb0bf1820689\") " pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.616588 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.616568 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q6bw\" (UniqueName: \"kubernetes.io/projected/10ca2f89-938d-4ed7-a81d-cf2f0f85d945-kube-api-access-5q6bw\") pod \"ovnkube-node-k9jjg\" (UID: \"10ca2f89-938d-4ed7-a81d-cf2f0f85d945\") " pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.617020 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.617001 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hct55\" (UniqueName: \"kubernetes.io/projected/d9f00b72-1079-4ead-a63f-5761b40d3166-kube-api-access-hct55\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:31.617270 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.617255 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bdc8\" (UniqueName: \"kubernetes.io/projected/155270d8-7663-4a7e-bc21-0af6009f4303-kube-api-access-5bdc8\") pod \"iptables-alerter-bv2hz\" (UID: \"155270d8-7663-4a7e-bc21-0af6009f4303\") " pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.617443 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.617430 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fj9k\" (UniqueName: \"kubernetes.io/projected/49901ffb-82bb-4c42-a126-2eef1b1c2b17-kube-api-access-6fj9k\") pod \"aws-ebs-csi-driver-node-jjrg7\" (UID: \"49901ffb-82bb-4c42-a126-2eef1b1c2b17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.701329 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.701286 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:31.709407 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.709388 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc8234a_907c_41cd_9900_f4183678e844.slice/crio-b271ed3c987f34f254251d7744ec6070fa327a583138ce9dbf7fad683babe117 WatchSource:0}: Error finding container b271ed3c987f34f254251d7744ec6070fa327a583138ce9dbf7fad683babe117: Status 404 returned error can't find the container with id b271ed3c987f34f254251d7744ec6070fa327a583138ce9dbf7fad683babe117 Jun 10 17:31:31.714446 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.714431 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2zd59" Jun 10 17:31:31.720854 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.720841 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n8cmd" Jun 10 17:31:31.721004 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.720984 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d5a1bc_53b2_4935_bcf0_23ffdb236dc8.slice/crio-ddf0fd3306c9016a17b703d8272a8e7c5c0ada9ad1d939509dc1c2131cdfc32f WatchSource:0}: Error finding container ddf0fd3306c9016a17b703d8272a8e7c5c0ada9ad1d939509dc1c2131cdfc32f: Status 404 returned error can't find the container with id ddf0fd3306c9016a17b703d8272a8e7c5c0ada9ad1d939509dc1c2131cdfc32f Jun 10 17:31:31.725005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.724987 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vsqlg" Jun 10 17:31:31.730345 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.729590 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hskvc" Jun 10 17:31:31.730345 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.729784 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b03051_fc01_4dac_a94b_b7ddc921fdf2.slice/crio-c483495987c3ea531be85c5f5b9a6b68517ef9eac3b4c494f0ffcd07484022be WatchSource:0}: Error finding container c483495987c3ea531be85c5f5b9a6b68517ef9eac3b4c494f0ffcd07484022be: Status 404 returned error can't find the container with id c483495987c3ea531be85c5f5b9a6b68517ef9eac3b4c494f0ffcd07484022be Jun 10 17:31:31.735756 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.735735 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bv2hz" Jun 10 17:31:31.735906 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.735808 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111e361e_8521_49d3_9685_9b21257ea3d3.slice/crio-954b89c495e05f7d3a84636909dc76257666c6136012c1483ad90eba71df032c WatchSource:0}: Error finding container 954b89c495e05f7d3a84636909dc76257666c6136012c1483ad90eba71df032c: Status 404 returned error can't find the container with id 954b89c495e05f7d3a84636909dc76257666c6136012c1483ad90eba71df032c Jun 10 17:31:31.738501 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.738480 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02074e9f_b06c_412e_beee_54138e398848.slice/crio-0a7210767b4350f732181d222ab274e1f5790f1ebd67248bc70834b4b41989d3 WatchSource:0}: Error finding container 0a7210767b4350f732181d222ab274e1f5790f1ebd67248bc70834b4b41989d3: Status 404 returned error can't find the container with id 0a7210767b4350f732181d222ab274e1f5790f1ebd67248bc70834b4b41989d3 Jun 10 17:31:31.741280 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.741258 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" Jun 10 17:31:31.743179 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.743158 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod155270d8_7663_4a7e_bc21_0af6009f4303.slice/crio-88bca3c2d543e7df9d35bc58821b70adb5841731edf19aed44485c79e2da00d3 WatchSource:0}: Error finding container 88bca3c2d543e7df9d35bc58821b70adb5841731edf19aed44485c79e2da00d3: Status 404 returned error can't find the container with id 88bca3c2d543e7df9d35bc58821b70adb5841731edf19aed44485c79e2da00d3 Jun 10 17:31:31.746362 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.746348 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" Jun 10 17:31:31.746859 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:31.746844 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:31.750192 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.750172 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49901ffb_82bb_4c42_a126_2eef1b1c2b17.slice/crio-f96459dbc77db16960714fdebd579bce28ba33a715b5a7a540dcb319ee4e90f1 WatchSource:0}: Error finding container f96459dbc77db16960714fdebd579bce28ba33a715b5a7a540dcb319ee4e90f1: Status 404 returned error can't find the container with id f96459dbc77db16960714fdebd579bce28ba33a715b5a7a540dcb319ee4e90f1 Jun 10 17:31:31.756472 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.756454 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c6b3a40_5850_42d2_b135_fb0bf1820689.slice/crio-5fbc9571fe199495998824d1c80185b1c707661af8936dc07efd3125d27f3d7e WatchSource:0}: Error finding container 5fbc9571fe199495998824d1c80185b1c707661af8936dc07efd3125d27f3d7e: Status 404 returned error can't find the container with id 5fbc9571fe199495998824d1c80185b1c707661af8936dc07efd3125d27f3d7e Jun 10 17:31:31.759461 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:31:31.759442 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ca2f89_938d_4ed7_a81d_cf2f0f85d945.slice/crio-ca78f0bc4f52c5e677990b02f26b91a7e491e1800d90b4909de66dc132dd2e52 WatchSource:0}: Error finding container ca78f0bc4f52c5e677990b02f26b91a7e491e1800d90b4909de66dc132dd2e52: Status 404 returned error can't find the container with id ca78f0bc4f52c5e677990b02f26b91a7e491e1800d90b4909de66dc132dd2e52 Jun 10 17:31:32.012303 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.012266 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:32.012472 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.012404 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 17:31:32.012472 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.012426 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 17:31:32.012472 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.012438 2531 projected.go:196] Error preparing data for projected volume kube-api-access-s7zq5 for pod openshift-network-diagnostics/network-check-target-8l6dz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:32.012612 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.012502 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5 podName:b15fa61d-010f-4536-9329-1b5b95efc2d4 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:33.012483836 +0000 UTC m=+2.980689858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s7zq5" (UniqueName: "kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5") pod "network-check-target-8l6dz" (UID: "b15fa61d-010f-4536-9329-1b5b95efc2d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:32.113203 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.113157 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:32.113361 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.113275 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:32.114515 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.113441 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:32.114515 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.113512 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:33.113483193 +0000 UTC m=+3.081689213 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:32.114515 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.113903 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:32.114515 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.113969 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret podName:70f889fb-10f8-4bc7-a0a2-42e0ab79dc87 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:33.113953754 +0000 UTC m=+3.082159781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret") pod "global-pull-secret-syncer-6wslj" (UID: "70f889fb-10f8-4bc7-a0a2-42e0ab79dc87") : object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:32.447357 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.447249 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 17:26:31 +0000 UTC" deadline="2028-03-01 20:39:33.266068582 +0000 UTC" Jun 10 17:31:32.447357 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.447311 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15123h8m0.818762478s" Jun 10 17:31:32.466431 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.465928 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:32.466431 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:32.466070 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:32.499442 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.499402 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" event={"ID":"9c6b3a40-5850-42d2-b135-fb0bf1820689","Type":"ContainerStarted","Data":"5fbc9571fe199495998824d1c80185b1c707661af8936dc07efd3125d27f3d7e"} Jun 10 17:31:32.511734 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.511702 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" event={"ID":"49901ffb-82bb-4c42-a126-2eef1b1c2b17","Type":"ContainerStarted","Data":"f96459dbc77db16960714fdebd579bce28ba33a715b5a7a540dcb319ee4e90f1"} Jun 10 17:31:32.522172 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.522142 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hskvc" event={"ID":"02074e9f-b06c-412e-beee-54138e398848","Type":"ContainerStarted","Data":"0a7210767b4350f732181d222ab274e1f5790f1ebd67248bc70834b4b41989d3"} Jun 10 17:31:32.533058 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.533025 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vsqlg" event={"ID":"111e361e-8521-49d3-9685-9b21257ea3d3","Type":"ContainerStarted","Data":"954b89c495e05f7d3a84636909dc76257666c6136012c1483ad90eba71df032c"} Jun 10 17:31:32.542881 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.542853 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zd59" event={"ID":"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8","Type":"ContainerStarted","Data":"ddf0fd3306c9016a17b703d8272a8e7c5c0ada9ad1d939509dc1c2131cdfc32f"} Jun 10 17:31:32.556617 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.556540 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rk5fq" event={"ID":"6bc8234a-907c-41cd-9900-f4183678e844","Type":"ContainerStarted","Data":"b271ed3c987f34f254251d7744ec6070fa327a583138ce9dbf7fad683babe117"} Jun 10 17:31:32.578546 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.578517 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"ca78f0bc4f52c5e677990b02f26b91a7e491e1800d90b4909de66dc132dd2e52"} Jun 10 17:31:32.592785 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.592755 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bv2hz" event={"ID":"155270d8-7663-4a7e-bc21-0af6009f4303","Type":"ContainerStarted","Data":"88bca3c2d543e7df9d35bc58821b70adb5841731edf19aed44485c79e2da00d3"} Jun 10 17:31:32.612877 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.612847 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n8cmd" event={"ID":"04b03051-fc01-4dac-a94b-b7ddc921fdf2","Type":"ContainerStarted","Data":"c483495987c3ea531be85c5f5b9a6b68517ef9eac3b4c494f0ffcd07484022be"} Jun 10 17:31:32.916037 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.916004 2531 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 17:31:32.917367 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:32.917343 2531 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 17:31:33.020650 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:33.020614 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:33.020821 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.020797 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 17:31:33.020821 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.020822 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 17:31:33.020953 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.020834 2531 projected.go:196] Error preparing data for projected volume kube-api-access-s7zq5 for pod openshift-network-diagnostics/network-check-target-8l6dz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:33.020953 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.020891 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5 podName:b15fa61d-010f-4536-9329-1b5b95efc2d4 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:35.020873376 +0000 UTC m=+4.989079407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s7zq5" (UniqueName: "kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5") pod "network-check-target-8l6dz" (UID: "b15fa61d-010f-4536-9329-1b5b95efc2d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:33.121741 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:33.121705 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:33.121918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:33.121773 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:33.122019 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.121993 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:33.122087 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.122061 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret podName:70f889fb-10f8-4bc7-a0a2-42e0ab79dc87 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:35.122041339 +0000 UTC m=+5.090247360 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret") pod "global-pull-secret-syncer-6wslj" (UID: "70f889fb-10f8-4bc7-a0a2-42e0ab79dc87") : object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:33.122457 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.122437 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:33.122532 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.122495 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:35.122479273 +0000 UTC m=+5.090685295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:33.447725 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:33.447676 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 17:26:31 +0000 UTC" deadline="2028-02-08 20:33:44.186947875 +0000 UTC" Jun 10 17:31:33.447725 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:33.447713 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14595h2m10.739238658s" Jun 10 17:31:33.464294 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:33.464270 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:33.464433 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.464404 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:33.464878 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:33.464856 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:33.464996 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:33.464973 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:33.530849 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:33.530821 2531 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 17:31:34.463898 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:34.463865 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:34.464358 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:34.464017 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:35.036632 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:35.036596 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:35.036832 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.036808 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 17:31:35.036910 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.036835 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 17:31:35.036910 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.036848 2531 projected.go:196] Error preparing data for projected volume kube-api-access-s7zq5 for pod openshift-network-diagnostics/network-check-target-8l6dz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:35.036910 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.036907 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5 podName:b15fa61d-010f-4536-9329-1b5b95efc2d4 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:39.036888245 +0000 UTC m=+9.005094265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s7zq5" (UniqueName: "kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5") pod "network-check-target-8l6dz" (UID: "b15fa61d-010f-4536-9329-1b5b95efc2d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:35.137785 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:35.137754 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:35.137974 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:35.137812 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:35.137974 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.137925 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:35.138091 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.137991 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:39.13797464 +0000 UTC m=+9.106180657 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:35.138303 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.138285 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:35.138372 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.138331 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret podName:70f889fb-10f8-4bc7-a0a2-42e0ab79dc87 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:39.138320266 +0000 UTC m=+9.106526283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret") pod "global-pull-secret-syncer-6wslj" (UID: "70f889fb-10f8-4bc7-a0a2-42e0ab79dc87") : object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:35.464159 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:35.464084 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:35.464581 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.464205 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:35.464581 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:35.464554 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:35.464699 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:35.464660 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:36.464112 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:36.464078 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:36.464294 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:36.464208 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:37.463775 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:37.463739 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:37.463978 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:37.463784 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:37.463978 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:37.463904 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:37.464115 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:37.464028 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:38.464460 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:38.464433 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:38.464919 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:38.464550 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:39.069100 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:39.069065 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:39.069280 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.069231 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 17:31:39.069280 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.069252 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 17:31:39.069280 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.069264 2531 projected.go:196] Error preparing data for projected volume kube-api-access-s7zq5 for pod openshift-network-diagnostics/network-check-target-8l6dz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:39.069445 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.069330 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5 podName:b15fa61d-010f-4536-9329-1b5b95efc2d4 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:47.069311987 +0000 UTC m=+17.037518007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s7zq5" (UniqueName: "kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5") pod "network-check-target-8l6dz" (UID: "b15fa61d-010f-4536-9329-1b5b95efc2d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:39.170288 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:39.170032 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:39.170288 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:39.170109 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:39.170288 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.170222 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:39.170288 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.170222 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:39.170288 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.170283 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret podName:70f889fb-10f8-4bc7-a0a2-42e0ab79dc87 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:47.170263973 +0000 UTC m=+17.138469989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret") pod "global-pull-secret-syncer-6wslj" (UID: "70f889fb-10f8-4bc7-a0a2-42e0ab79dc87") : object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:39.170707 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.170300 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:31:47.170290442 +0000 UTC m=+17.138496464 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:39.464322 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:39.464243 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:39.464322 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:39.464260 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:39.464531 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.464373 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:39.464834 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:39.464522 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:40.464461 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:40.464432 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:40.464620 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:40.464540 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:41.464375 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:41.464331 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:41.464375 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:41.464369 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:41.464601 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:41.464466 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:41.464601 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:41.464587 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:42.464133 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:42.464097 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:42.464319 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:42.464231 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:43.464227 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:43.464199 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:43.464227 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:43.464225 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:43.464694 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:43.464299 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:43.464694 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:43.464442 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:44.463804 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:44.463757 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:44.463975 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:44.463881 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:45.463837 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:45.463741 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:45.464251 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:45.463867 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:45.464251 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:45.463914 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:45.464251 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:45.464042 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:46.464091 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:46.464042 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:46.464486 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:46.464183 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:47.130311 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:47.130275 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:47.130504 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.130483 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 17:31:47.130551 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.130512 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 17:31:47.130551 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.130526 2531 projected.go:196] Error preparing data for projected volume kube-api-access-s7zq5 for pod openshift-network-diagnostics/network-check-target-8l6dz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:47.130632 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.130590 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5 podName:b15fa61d-010f-4536-9329-1b5b95efc2d4 nodeName:}" failed. No retries permitted until 2026-06-10 17:32:03.130570591 +0000 UTC m=+33.098776609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-s7zq5" (UniqueName: "kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5") pod "network-check-target-8l6dz" (UID: "b15fa61d-010f-4536-9329-1b5b95efc2d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:31:47.231511 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:47.231485 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:47.231680 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:47.231534 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:47.231680 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.231617 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:47.231680 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.231620 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:47.231680 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.231663 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:32:03.231649417 +0000 UTC m=+33.199855433 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:31:47.231680 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.231675 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret podName:70f889fb-10f8-4bc7-a0a2-42e0ab79dc87 nodeName:}" failed. No retries permitted until 2026-06-10 17:32:03.231669543 +0000 UTC m=+33.199875559 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret") pod "global-pull-secret-syncer-6wslj" (UID: "70f889fb-10f8-4bc7-a0a2-42e0ab79dc87") : object "kube-system"/"original-pull-secret" not registered Jun 10 17:31:47.464179 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:47.464152 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:47.464531 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:47.464190 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:47.464531 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.464267 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:47.464531 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:47.464400 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:48.463854 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:48.463820 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:48.464032 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:48.463956 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:49.464137 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:49.464040 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:49.464702 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:49.464175 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:49.464702 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:49.464238 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:49.464702 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:49.464347 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:50.465062 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:50.465032 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:50.465465 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:50.465136 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:51.464065 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:51.464025 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:51.464238 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:51.464026 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:51.464238 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:51.464152 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:51.464358 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:51.464297 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:52.464080 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.463829 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:52.464613 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:52.464181 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:52.667049 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.666862 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:31:52.667446 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.667422 2531 generic.go:356] "Generic (PLEG): container finished" podID="10ca2f89-938d-4ed7-a81d-cf2f0f85d945" containerID="a179165b2632d8da9af1df4807e0ead0c1279c164d3bf295cb022148c5e0e921" exitCode=1 Jun 10 17:31:52.667531 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.667498 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"9ec51dcadbe91c857be85c1a4027a45fa48d8a628fcbacdf8b29b3135ce05bec"} Jun 10 17:31:52.667620 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.667597 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerDied","Data":"a179165b2632d8da9af1df4807e0ead0c1279c164d3bf295cb022148c5e0e921"} Jun 10 17:31:52.667729 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.667666 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"945aeb2e33d2451fb4c172259c156b7372787a5239d41802c0d29551fb922e91"} Jun 10 17:31:52.669433 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.669406 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" event={"ID":"9c6b3a40-5850-42d2-b135-fb0bf1820689","Type":"ContainerStarted","Data":"dd92506ce2093c2bac54a8681a837b12ea32c7160a0925347786262716a46e5b"} Jun 10 17:31:52.670985 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.670964 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hskvc" event={"ID":"02074e9f-b06c-412e-beee-54138e398848","Type":"ContainerStarted","Data":"c57d6fbf201d0f6a033af4a3821458c4800b92e010b9f360633111fc29725c4c"} Jun 10 17:31:52.672501 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.672477 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" event={"ID":"193fc2f766e2d337515229dea4784854","Type":"ContainerStarted","Data":"b900723e2d08cf5198e835dbf465b02990b822bf6cc1caea09f4c4f55fce9aab"} Jun 10 17:31:52.699144 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.699107 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" podStartSLOduration=21.699092909 podStartE2EDuration="21.699092909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:52.698880463 +0000 UTC m=+22.667086501" watchObservedRunningTime="2026-06-10 17:31:52.699092909 +0000 UTC m=+22.667298948" Jun 10 17:31:52.699261 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:52.699244 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z5xvj" podStartSLOduration=2.441348436 podStartE2EDuration="22.699238209s" totalImagesPullingTime="20.257889773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:52.685993664 +0000 UTC m=+22.654199700" watchObservedRunningTime="2026-06-10 17:31:52.699238209 +0000 UTC m=+22.667444248" Jun 10 17:31:53.464103 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.464073 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:53.464575 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.464073 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:53.464575 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:53.464476 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:53.464973 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:53.464646 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:53.676473 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.676395 2531 generic.go:356] "Generic (PLEG): container finished" podID="281b45fad3c7c003f8a039bb575ccb10" containerID="5f9e3dc437091b074d3413eef340509086d7246da245a9e7dc70b5eafee8a537" exitCode=0 Jun 10 17:31:53.676606 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.676475 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" event={"ID":"281b45fad3c7c003f8a039bb575ccb10","Type":"ContainerDied","Data":"5f9e3dc437091b074d3413eef340509086d7246da245a9e7dc70b5eafee8a537"} Jun 10 17:31:53.677953 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.677912 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" event={"ID":"49901ffb-82bb-4c42-a126-2eef1b1c2b17","Type":"ContainerStarted","Data":"2cba5f6d9b3b3869fb0b6c51444b74961f16eece16c6f8bed1b0fd8dc5f9b02d"} Jun 10 17:31:53.679296 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.679279 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vsqlg" event={"ID":"111e361e-8521-49d3-9685-9b21257ea3d3","Type":"ContainerStarted","Data":"36bba086c5a2c34272d5abfb07858a75fd0cd7e14d8d63f6f1d5e07c9690ec22"} Jun 10 17:31:53.680735 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.680712 2531 generic.go:356] "Generic (PLEG): container finished" podID="f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8" containerID="db4132d43ee238cbf3c517b8f1aa9236a0611840519b7c7276c96b45f4a8bbf2" exitCode=0 Jun 10 17:31:53.680842 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.680738 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zd59" event={"ID":"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8","Type":"ContainerDied","Data":"db4132d43ee238cbf3c517b8f1aa9236a0611840519b7c7276c96b45f4a8bbf2"} Jun 10 17:31:53.682306 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.682291 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rk5fq" event={"ID":"6bc8234a-907c-41cd-9900-f4183678e844","Type":"ContainerStarted","Data":"22803fe528d85f315361899b1d3fa192dc69ac94dbb855e9e9f421944506380a"} Jun 10 17:31:53.685024 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.685008 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:31:53.685345 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.685329 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"505c51fff077c4d404a032477c7b0b761cbc5b5cefd3ea1803e4cfff02103528"} Jun 10 17:31:53.685400 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.685386 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"a8fe0e7fa61debfa0ffead2e25f4e43115fbca48584c325166491fdfee3554a2"} Jun 10 17:31:53.685461 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.685446 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"846f9f7c1e35d35954bda696f217ab310a83968e14e7991f01591d8c1dd1ab21"} Jun 10 17:31:53.686746 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.686728 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bv2hz" event={"ID":"155270d8-7663-4a7e-bc21-0af6009f4303","Type":"ContainerStarted","Data":"2381f07c6e948213cda816a4352d37a08cc9b909547f298e3f6ae1c769279c7d"} Jun 10 17:31:53.688042 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.688023 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n8cmd" event={"ID":"04b03051-fc01-4dac-a94b-b7ddc921fdf2","Type":"ContainerStarted","Data":"245a25e08672c5c549ca8c1b836c4ca0f02c0518e8692ed596b04693feb9bd4a"} Jun 10 17:31:53.691989 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.691931 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hskvc" podStartSLOduration=3.417250712 podStartE2EDuration="23.691917135s" totalImagesPullingTime="20.274666423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:52.714229741 +0000 UTC m=+22.682435779" watchObservedRunningTime="2026-06-10 17:31:53.691917135 +0000 UTC m=+23.660123173" Jun 10 17:31:53.692582 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.692549 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vsqlg" podStartSLOduration=3.844363399 podStartE2EDuration="23.692536827s" totalImagesPullingTime="19.848173428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:53.692058754 +0000 UTC m=+23.660264774" watchObservedRunningTime="2026-06-10 17:31:53.692536827 +0000 UTC m=+23.660742868" Jun 10 17:31:53.704906 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.704873 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bv2hz" podStartSLOduration=3.460853381 podStartE2EDuration="23.704862556s" totalImagesPullingTime="20.244009175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:53.704451252 +0000 UTC m=+23.672657290" watchObservedRunningTime="2026-06-10 17:31:53.704862556 +0000 UTC m=+23.673068594" Jun 10 17:31:53.717473 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.717438 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rk5fq" podStartSLOduration=3.437760687 podStartE2EDuration="23.717427075s" totalImagesPullingTime="20.279666388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:53.717170451 +0000 UTC m=+23.685376490" watchObservedRunningTime="2026-06-10 17:31:53.717427075 +0000 UTC m=+23.685633113" Jun 10 17:31:53.751851 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:53.751813 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n8cmd" podStartSLOduration=3.465194799 podStartE2EDuration="23.751802025s" totalImagesPullingTime="20.286607226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:53.751357938 +0000 UTC m=+23.719563977" watchObservedRunningTime="2026-06-10 17:31:53.751802025 +0000 UTC m=+23.720008064" Jun 10 17:31:54.464443 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:54.464156 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:54.464859 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:54.464583 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:54.548292 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:54.548265 2531 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 10 17:31:54.692206 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:54.692120 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" event={"ID":"49901ffb-82bb-4c42-a126-2eef1b1c2b17","Type":"ContainerStarted","Data":"e320d83825a1adb0eff617d4b1128d4ecd5f1b582f0531f313ecf78b4c74ec89"} Jun 10 17:31:54.951111 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:54.951041 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:54.951710 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:54.951689 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:55.464302 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.464274 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:55.464587 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.464274 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:55.464587 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:55.464388 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:55.464587 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:55.464462 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:55.473384 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.473302 2531 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-10T17:31:54.548289106Z","UUID":"2e43800b-b126-4c0d-9995-90cf8fc82530","Handler":null,"Name":"","Endpoint":""} Jun 10 17:31:55.474885 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.474862 2531 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 10 17:31:55.475014 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.474903 2531 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 10 17:31:55.698025 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.697989 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" event={"ID":"281b45fad3c7c003f8a039bb575ccb10","Type":"ContainerStarted","Data":"c00653476e9c3a7c949864264087397e6535a21103fc98e2561fbe86abbaf6e3"} Jun 10 17:31:55.701475 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.701451 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:31:55.701930 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.701903 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"29fd648ff017fb59c2dc3a979688c3a266fbbaac81a83a7a09c2b5f6bf6b5967"} Jun 10 17:31:55.702304 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.702288 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:55.702705 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.702688 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rk5fq" Jun 10 17:31:55.712079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:55.712035 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" podStartSLOduration=24.712012901 podStartE2EDuration="24.712012901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:55.711218069 +0000 UTC m=+25.679424108" watchObservedRunningTime="2026-06-10 17:31:55.712012901 +0000 UTC m=+25.680218944" Jun 10 17:31:56.464624 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:56.464594 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:56.465117 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:56.464700 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:56.706658 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:56.706607 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" event={"ID":"49901ffb-82bb-4c42-a126-2eef1b1c2b17","Type":"ContainerStarted","Data":"3f4ec2e5fd6ae6092e723ca98240d2f54b6ab427172e7bd2c6260e6272af106a"} Jun 10 17:31:56.723416 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:56.723220 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jjrg7" podStartSLOduration=4.347737401 podStartE2EDuration="26.723199391s" totalImagesPullingTime="22.37546199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:56.722338576 +0000 UTC m=+26.690544633" watchObservedRunningTime="2026-06-10 17:31:56.723199391 +0000 UTC m=+26.691405424" Jun 10 17:31:57.463717 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:57.463640 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:57.463876 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:57.463767 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:57.463876 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:57.463776 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:57.463876 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:57.463868 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:58.464283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.464137 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:31:58.464588 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:58.464367 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:31:58.712500 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.712430 2531 generic.go:356] "Generic (PLEG): container finished" podID="f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8" containerID="6554d274fbec8a764ed27e1eb206bfc98c1be955980da647660f623f504c55d8" exitCode=0 Jun 10 17:31:58.712646 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.712518 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zd59" event={"ID":"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8","Type":"ContainerDied","Data":"6554d274fbec8a764ed27e1eb206bfc98c1be955980da647660f623f504c55d8"} Jun 10 17:31:58.716037 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.716023 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:31:58.716378 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.716344 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"0c4dc5e49b243316c7331481b601565a1888651edec4f4f715d560cd0f9640c3"} Jun 10 17:31:58.716642 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.716618 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:58.716733 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.716709 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:58.716840 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.716825 2531 scope.go:117] "RemoveContainer" containerID="a179165b2632d8da9af1df4807e0ead0c1279c164d3bf295cb022148c5e0e921" Jun 10 17:31:58.733324 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:58.733307 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:59.464058 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:59.464030 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:31:59.464256 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:59.464134 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:31:59.464256 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:59.464182 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:31:59.464527 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:31:59.464285 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:31:59.722958 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:59.722873 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:31:59.723238 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:59.723214 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" event={"ID":"10ca2f89-938d-4ed7-a81d-cf2f0f85d945","Type":"ContainerStarted","Data":"ae3adc9ee189136735a53b7386fd9aac971a89bc78cde1e350e67cc67d40e1bd"} Jun 10 17:31:59.723405 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:59.723385 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:59.740288 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:59.740268 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:31:59.751447 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:31:59.751393 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" podStartSLOduration=9.228297712 podStartE2EDuration="29.751374241s" totalImagesPullingTime="20.523076529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:31:59.74967656 +0000 UTC m=+29.717882599" watchObservedRunningTime="2026-06-10 17:31:59.751374241 +0000 UTC m=+29.719580280" Jun 10 17:32:00.118049 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:00.118013 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6wslj"] Jun 10 17:32:00.118201 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:00.118141 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:32:00.118249 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:00.118226 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:32:00.121385 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:00.121360 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f5f66"] Jun 10 17:32:00.121555 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:00.121464 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:32:00.121632 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:00.121572 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:32:00.121958 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:00.121923 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8l6dz"] Jun 10 17:32:00.122041 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:00.122020 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:00.122139 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:00.122106 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:32:00.727567 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:00.727532 2531 generic.go:356] "Generic (PLEG): container finished" podID="f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8" containerID="b70c3db3d8e00fbc32420eecd1855bcbafbeae3f61839eadb5553b7f7f723808" exitCode=0 Jun 10 17:32:00.728384 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:00.727616 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zd59" event={"ID":"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8","Type":"ContainerDied","Data":"b70c3db3d8e00fbc32420eecd1855bcbafbeae3f61839eadb5553b7f7f723808"} Jun 10 17:32:01.463651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:01.463579 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:01.463794 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:01.463682 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:32:01.463794 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:01.463590 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:32:01.463794 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:01.463715 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:32:01.463925 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:01.463830 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:32:01.464057 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:01.463917 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:32:02.733783 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:02.733605 2531 generic.go:356] "Generic (PLEG): container finished" podID="f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8" containerID="6c190764f95819d238879119ebf6b3d32951068611e464beda020fcdea0525bd" exitCode=0 Jun 10 17:32:02.733783 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:02.733686 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zd59" event={"ID":"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8","Type":"ContainerDied","Data":"6c190764f95819d238879119ebf6b3d32951068611e464beda020fcdea0525bd"} Jun 10 17:32:03.153319 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:03.153282 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:03.153484 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.153430 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 17:32:03.153484 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.153449 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 17:32:03.153484 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.153458 2531 projected.go:196] Error preparing data for projected volume kube-api-access-s7zq5 for pod openshift-network-diagnostics/network-check-target-8l6dz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:32:03.153591 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.153505 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5 podName:b15fa61d-010f-4536-9329-1b5b95efc2d4 nodeName:}" failed. No retries permitted until 2026-06-10 17:32:35.153491621 +0000 UTC m=+65.121697638 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-s7zq5" (UniqueName: "kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5") pod "network-check-target-8l6dz" (UID: "b15fa61d-010f-4536-9329-1b5b95efc2d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 17:32:03.253794 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:03.253766 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:32:03.253972 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:03.253822 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:32:03.253972 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.253923 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 17:32:03.253972 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.253926 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:32:03.254140 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.254006 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret podName:70f889fb-10f8-4bc7-a0a2-42e0ab79dc87 nodeName:}" failed. No retries permitted until 2026-06-10 17:32:35.253986894 +0000 UTC m=+65.222192929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret") pod "global-pull-secret-syncer-6wslj" (UID: "70f889fb-10f8-4bc7-a0a2-42e0ab79dc87") : object "kube-system"/"original-pull-secret" not registered Jun 10 17:32:03.254140 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.254027 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:32:35.254015922 +0000 UTC m=+65.222221944 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 17:32:03.465356 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:03.465260 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:32:03.465356 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:03.465297 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:03.465356 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:03.465305 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:32:03.465574 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.465397 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6wslj" podUID="70f889fb-10f8-4bc7-a0a2-42e0ab79dc87" Jun 10 17:32:03.465574 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.465480 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l6dz" podUID="b15fa61d-010f-4536-9329-1b5b95efc2d4" Jun 10 17:32:03.465574 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:03.465572 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:32:05.393214 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.393183 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeReady" Jun 10 17:32:05.393641 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.393316 2531 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 10 17:32:05.437121 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.437079 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2227b"] Jun 10 17:32:05.469871 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.469835 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tw5jz"] Jun 10 17:32:05.470064 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.469892 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:05.470064 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.469913 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:32:05.470064 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.469918 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.470064 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.469922 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:32:05.473016 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.472993 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 17:32:05.473403 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.473386 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 17:32:05.473644 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.473619 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dtfgg\"" Jun 10 17:32:05.473722 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.473654 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dqnwj\"" Jun 10 17:32:05.474087 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.474061 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 17:32:05.474175 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.474031 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 10 17:32:05.475781 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.475605 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d9nks\"" Jun 10 17:32:05.477134 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.477111 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 17:32:05.477455 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.477434 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 10 17:32:05.486149 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.486132 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tw5jz"] Jun 10 17:32:05.486242 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.486158 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2227b"] Jun 10 17:32:05.486242 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.486220 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tw5jz" Jun 10 17:32:05.489336 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.489320 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gd7vx\"" Jun 10 17:32:05.489651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.489637 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 10 17:32:05.489827 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.489806 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 10 17:32:05.490260 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.490243 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 10 17:32:05.572846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.572817 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxm8m\" (UniqueName: \"kubernetes.io/projected/6f80cc11-56c3-4148-bd3f-17559675eaed-kube-api-access-gxm8m\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.573011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.572886 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvbqc\" (UniqueName: \"kubernetes.io/projected/6fc5ad1d-f684-4158-b1e1-bac37539d5f9-kube-api-access-vvbqc\") pod \"ingress-canary-tw5jz\" (UID: \"6fc5ad1d-f684-4158-b1e1-bac37539d5f9\") " pod="openshift-ingress-canary/ingress-canary-tw5jz" Jun 10 17:32:05.573011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.572970 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6f80cc11-56c3-4148-bd3f-17559675eaed-config-volume\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.573011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.573001 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fc5ad1d-f684-4158-b1e1-bac37539d5f9-cert\") pod \"ingress-canary-tw5jz\" (UID: \"6fc5ad1d-f684-4158-b1e1-bac37539d5f9\") " pod="openshift-ingress-canary/ingress-canary-tw5jz" Jun 10 17:32:05.573149 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.573014 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f80cc11-56c3-4148-bd3f-17559675eaed-tmp-dir\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.573149 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.573030 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.673748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.673656 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6f80cc11-56c3-4148-bd3f-17559675eaed-config-volume\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.673748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.673718 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fc5ad1d-f684-4158-b1e1-bac37539d5f9-cert\") pod \"ingress-canary-tw5jz\" (UID: \"6fc5ad1d-f684-4158-b1e1-bac37539d5f9\") " pod="openshift-ingress-canary/ingress-canary-tw5jz" Jun 10 17:32:05.673748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.673736 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f80cc11-56c3-4148-bd3f-17559675eaed-tmp-dir\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.674053 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.673764 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.674053 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.673800 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxm8m\" (UniqueName: \"kubernetes.io/projected/6f80cc11-56c3-4148-bd3f-17559675eaed-kube-api-access-gxm8m\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.674053 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.673833 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvbqc\" (UniqueName: \"kubernetes.io/projected/6fc5ad1d-f684-4158-b1e1-bac37539d5f9-kube-api-access-vvbqc\") pod \"ingress-canary-tw5jz\" (UID: \"6fc5ad1d-f684-4158-b1e1-bac37539d5f9\") " pod="openshift-ingress-canary/ingress-canary-tw5jz" Jun 10 17:32:05.674311 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.674287 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6f80cc11-56c3-4148-bd3f-17559675eaed-config-volume\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.674518 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.674490 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f80cc11-56c3-4148-bd3f-17559675eaed-tmp-dir\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.674597 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:05.674497 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:32:05.674597 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:05.674575 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:32:06.174554882 +0000 UTC m=+36.142760899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:32:05.678610 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.678587 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fc5ad1d-f684-4158-b1e1-bac37539d5f9-cert\") pod \"ingress-canary-tw5jz\" (UID: \"6fc5ad1d-f684-4158-b1e1-bac37539d5f9\") " pod="openshift-ingress-canary/ingress-canary-tw5jz" Jun 10 17:32:05.682366 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.682343 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvbqc\" (UniqueName: \"kubernetes.io/projected/6fc5ad1d-f684-4158-b1e1-bac37539d5f9-kube-api-access-vvbqc\") pod \"ingress-canary-tw5jz\" (UID: \"6fc5ad1d-f684-4158-b1e1-bac37539d5f9\") " pod="openshift-ingress-canary/ingress-canary-tw5jz" Jun 10 17:32:05.682455 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.682402 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxm8m\" (UniqueName: \"kubernetes.io/projected/6f80cc11-56c3-4148-bd3f-17559675eaed-kube-api-access-gxm8m\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:05.812700 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:05.812650 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tw5jz" Jun 10 17:32:06.178978 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:06.178921 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:06.179140 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:06.179072 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:32:06.179206 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:06.179153 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:32:07.179132597 +0000 UTC m=+37.147338617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:32:06.967563 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:06.967533 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tw5jz"] Jun 10 17:32:06.977613 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:32:06.977582 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc5ad1d_f684_4158_b1e1_bac37539d5f9.slice/crio-0ad3ba66d8ebdb13efa9b7d70ab0fd812d3db5470c6e198af308179554ef1ca0 WatchSource:0}: Error finding container 0ad3ba66d8ebdb13efa9b7d70ab0fd812d3db5470c6e198af308179554ef1ca0: Status 404 returned error can't find the container with id 0ad3ba66d8ebdb13efa9b7d70ab0fd812d3db5470c6e198af308179554ef1ca0 Jun 10 17:32:07.185980 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:07.185792 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:07.186113 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:07.185952 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:32:07.186113 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:07.186049 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:32:09.186033205 +0000 UTC m=+39.154239222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:32:07.746987 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:07.746928 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tw5jz" event={"ID":"6fc5ad1d-f684-4158-b1e1-bac37539d5f9","Type":"ContainerStarted","Data":"0ad3ba66d8ebdb13efa9b7d70ab0fd812d3db5470c6e198af308179554ef1ca0"} Jun 10 17:32:07.750108 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:07.750002 2531 generic.go:356] "Generic (PLEG): container finished" podID="f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8" containerID="3a9cc76f2aca92da12c124a4f3d282b59081f88cdfe37ce095fe8e8560d82f99" exitCode=0 Jun 10 17:32:07.750108 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:07.750060 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zd59" event={"ID":"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8","Type":"ContainerDied","Data":"3a9cc76f2aca92da12c124a4f3d282b59081f88cdfe37ce095fe8e8560d82f99"} Jun 10 17:32:08.755895 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:08.755843 2531 generic.go:356] "Generic (PLEG): container finished" podID="f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8" containerID="ba9ca42c077bab242d765994814d77bc4aa50d8c9b823a63b45d191331ab243e" exitCode=0 Jun 10 17:32:08.755895 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:08.755888 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zd59" event={"ID":"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8","Type":"ContainerDied","Data":"ba9ca42c077bab242d765994814d77bc4aa50d8c9b823a63b45d191331ab243e"} Jun 10 17:32:09.201722 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:09.201628 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:09.201871 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:09.201808 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:32:09.201958 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:09.201890 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:32:13.201864945 +0000 UTC m=+43.170070969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:32:09.762233 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:09.762203 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2zd59" event={"ID":"f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8","Type":"ContainerStarted","Data":"3e09d9b77f954426aebbbf694cd164ad666e1f83895bb866ff75be2f664870c9"} Jun 10 17:32:09.785916 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:09.785864 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2zd59" podStartSLOduration=8.562373750999999 podStartE2EDuration="39.785836243s" totalImagesPullingTime="31.223462492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:32:09.784626008 +0000 UTC m=+39.752832089" watchObservedRunningTime="2026-06-10 17:32:09.785836243 +0000 UTC m=+39.754042283" Jun 10 17:32:10.765924 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:10.765882 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tw5jz" event={"ID":"6fc5ad1d-f684-4158-b1e1-bac37539d5f9","Type":"ContainerStarted","Data":"66937b8de63dc968c5be4e2473a90edf7baae5a8f994db8a459c4ca3c247a89a"} Jun 10 17:32:10.780804 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:10.780752 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tw5jz" podStartSLOduration=3.029575275 podStartE2EDuration="5.78073882s" totalImagesPullingTime="2.751163545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:32:10.779776851 +0000 UTC m=+40.747982912" watchObservedRunningTime="2026-06-10 17:32:10.78073882 +0000 UTC m=+40.748944859" Jun 10 17:32:13.228153 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:13.228114 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:13.228506 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:13.228263 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:32:13.228506 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:13.228331 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:32:21.228314456 +0000 UTC m=+51.196520474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:32:19.008107 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.008072 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x"] Jun 10 17:32:19.031304 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.031276 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267"] Jun 10 17:32:19.031447 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.031407 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.033713 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.033689 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 10 17:32:19.033713 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.033709 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 10 17:32:19.034039 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.033776 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 10 17:32:19.034039 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.033864 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 10 17:32:19.034039 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.033880 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 10 17:32:19.034039 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.034014 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 10 17:32:19.034233 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.034211 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 10 17:32:19.047378 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.047359 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267"] Jun 10 17:32:19.047474 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.047387 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x"] Jun 10 17:32:19.047474 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.047460 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.049391 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.049373 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 10 17:32:19.167313 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167269 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9f25\" (UniqueName: \"kubernetes.io/projected/3944211e-a2f9-4c8e-98a2-689228b4cd9f-kube-api-access-s9f25\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.167313 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167313 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.167528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167330 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.167528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167364 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-658rm\" (UniqueName: \"kubernetes.io/projected/b2fc77d5-50ae-4fba-882d-3c9f43931519-kube-api-access-658rm\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.167528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167396 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2fc77d5-50ae-4fba-882d-3c9f43931519-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.167528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167411 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3944211e-a2f9-4c8e-98a2-689228b4cd9f-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.167528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167467 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-ca\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.167528 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167511 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-hub\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.167732 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.167548 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3944211e-a2f9-4c8e-98a2-689228b4cd9f-tmp\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.268634 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268557 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9f25\" (UniqueName: \"kubernetes.io/projected/3944211e-a2f9-4c8e-98a2-689228b4cd9f-kube-api-access-s9f25\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.268634 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268598 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.268634 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268623 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.268882 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268664 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-658rm\" (UniqueName: \"kubernetes.io/projected/b2fc77d5-50ae-4fba-882d-3c9f43931519-kube-api-access-658rm\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.268882 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268713 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2fc77d5-50ae-4fba-882d-3c9f43931519-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.268882 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268738 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3944211e-a2f9-4c8e-98a2-689228b4cd9f-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.268882 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268864 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-ca\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.269111 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268913 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-hub\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.269111 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.268978 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3944211e-a2f9-4c8e-98a2-689228b4cd9f-tmp\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.269386 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.269356 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3944211e-a2f9-4c8e-98a2-689228b4cd9f-tmp\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.269502 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.269445 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2fc77d5-50ae-4fba-882d-3c9f43931519-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.272514 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.272483 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3944211e-a2f9-4c8e-98a2-689228b4cd9f-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.272618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.272551 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.272618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.272556 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-ca\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.272618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.272567 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.272618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.272587 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b2fc77d5-50ae-4fba-882d-3c9f43931519-hub\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.275723 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.275701 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9f25\" (UniqueName: \"kubernetes.io/projected/3944211e-a2f9-4c8e-98a2-689228b4cd9f-kube-api-access-s9f25\") pod \"klusterlet-addon-workmgr-6b9855bfc7-dr267\" (UID: \"3944211e-a2f9-4c8e-98a2-689228b4cd9f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.276219 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.276199 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-658rm\" (UniqueName: \"kubernetes.io/projected/b2fc77d5-50ae-4fba-882d-3c9f43931519-kube-api-access-658rm\") pod \"cluster-proxy-proxy-agent-6b68464d89-d5g6x\" (UID: \"b2fc77d5-50ae-4fba-882d-3c9f43931519\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.349580 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.349556 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:32:19.355631 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.355613 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:19.529957 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.529861 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x"] Jun 10 17:32:19.532769 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:32:19.532740 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2fc77d5_50ae_4fba_882d_3c9f43931519.slice/crio-3fc2aa5ef23040b1bdbd3eb8f33cbda076f386ee59b77b7be3aeade971ee7410 WatchSource:0}: Error finding container 3fc2aa5ef23040b1bdbd3eb8f33cbda076f386ee59b77b7be3aeade971ee7410: Status 404 returned error can't find the container with id 3fc2aa5ef23040b1bdbd3eb8f33cbda076f386ee59b77b7be3aeade971ee7410 Jun 10 17:32:19.556775 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.556753 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267"] Jun 10 17:32:19.559260 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:32:19.559239 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3944211e_a2f9_4c8e_98a2_689228b4cd9f.slice/crio-0123543db1e84580b7225de88bc467d454cb7d01fd0b38b87f1a6bdc57fb55d0 WatchSource:0}: Error finding container 0123543db1e84580b7225de88bc467d454cb7d01fd0b38b87f1a6bdc57fb55d0: Status 404 returned error can't find the container with id 0123543db1e84580b7225de88bc467d454cb7d01fd0b38b87f1a6bdc57fb55d0 Jun 10 17:32:19.789089 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.789005 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" event={"ID":"b2fc77d5-50ae-4fba-882d-3c9f43931519","Type":"ContainerStarted","Data":"3fc2aa5ef23040b1bdbd3eb8f33cbda076f386ee59b77b7be3aeade971ee7410"} Jun 10 17:32:19.790051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:19.790016 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" event={"ID":"3944211e-a2f9-4c8e-98a2-689228b4cd9f","Type":"ContainerStarted","Data":"0123543db1e84580b7225de88bc467d454cb7d01fd0b38b87f1a6bdc57fb55d0"} Jun 10 17:32:21.285157 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:21.285117 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:21.285624 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:21.285291 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:32:21.285624 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:21.285366 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:32:37.285344464 +0000 UTC m=+67.253550481 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:32:25.808430 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:25.807795 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" event={"ID":"3944211e-a2f9-4c8e-98a2-689228b4cd9f","Type":"ContainerStarted","Data":"2d5cd1e193b107b5f047ea37a95e11e99e1432cb948da0b15b31724b09b92710"} Jun 10 17:32:25.808430 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:25.808421 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:25.810462 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:25.809710 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:32:25.813907 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:25.813886 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" event={"ID":"b2fc77d5-50ae-4fba-882d-3c9f43931519","Type":"ContainerStarted","Data":"d7e86f4a67f4b1d195ae8f773a5eeb46cc32ea517c825a53f7a1b2f49f1ccc1c"} Jun 10 17:32:25.813998 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:25.813920 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" event={"ID":"b2fc77d5-50ae-4fba-882d-3c9f43931519","Type":"ContainerStarted","Data":"d3483178917cf839e178e97341c5b4f00c6919e3e624359a7fb7e4f2cb6b7c71"} Jun 10 17:32:25.824415 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:25.824371 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" podStartSLOduration=1.716538318 podStartE2EDuration="7.824359313s" totalImagesPullingTime="6.107820995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:32:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:32:25.823559547 +0000 UTC m=+55.791765586" watchObservedRunningTime="2026-06-10 17:32:25.824359313 +0000 UTC m=+55.792565365" Jun 10 17:32:26.818718 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:26.818678 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" event={"ID":"b2fc77d5-50ae-4fba-882d-3c9f43931519","Type":"ContainerStarted","Data":"04533245ae762a796cc951566b951010d46edc5e25069c2aa01c5da2fd07491a"} Jun 10 17:32:26.837348 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:26.837305 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" podStartSLOduration=2.704463656 podStartE2EDuration="8.83729037s" totalImagesPullingTime="6.132826714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:32:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:32:26.835979911 +0000 UTC m=+56.804185949" watchObservedRunningTime="2026-06-10 17:32:26.83729037 +0000 UTC m=+56.805496421" Jun 10 17:32:31.740618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:31.740592 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k9jjg" Jun 10 17:32:35.185814 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.185765 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:35.188157 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.188139 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 17:32:35.198516 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.198497 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 17:32:35.209489 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.209463 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7zq5\" (UniqueName: \"kubernetes.io/projected/b15fa61d-010f-4536-9329-1b5b95efc2d4-kube-api-access-s7zq5\") pod \"network-check-target-8l6dz\" (UID: \"b15fa61d-010f-4536-9329-1b5b95efc2d4\") " pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:35.286614 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.286581 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:32:35.286775 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.286634 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:32:35.288748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.288725 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 17:32:35.288861 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.288798 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 17:32:35.297499 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:35.297480 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 17:32:35.297572 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:35.297545 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:33:39.297525581 +0000 UTC m=+129.265731598 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : secret "metrics-daemon-secret" not found Jun 10 17:32:35.299640 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.299622 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70f889fb-10f8-4bc7-a0a2-42e0ab79dc87-original-pull-secret\") pod \"global-pull-secret-syncer-6wslj\" (UID: \"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87\") " pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:32:35.487331 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.487250 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dtfgg\"" Jun 10 17:32:35.495978 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.495958 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:35.506541 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.506447 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6wslj" Jun 10 17:32:35.629791 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.629709 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8l6dz"] Jun 10 17:32:35.634263 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:32:35.634232 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15fa61d_010f_4536_9329_1b5b95efc2d4.slice/crio-6c1e2729bb3618d00d0133d310fbc595e00d5cfdc1664c1ddaca43cfdf2f16cc WatchSource:0}: Error finding container 6c1e2729bb3618d00d0133d310fbc595e00d5cfdc1664c1ddaca43cfdf2f16cc: Status 404 returned error can't find the container with id 6c1e2729bb3618d00d0133d310fbc595e00d5cfdc1664c1ddaca43cfdf2f16cc Jun 10 17:32:35.646118 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.646091 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6wslj"] Jun 10 17:32:35.649508 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:32:35.649482 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f889fb_10f8_4bc7_a0a2_42e0ab79dc87.slice/crio-e8d522a5b9f1438a72e758570ecd054f15a3ecff3297bb5921fe229df2ba4beb WatchSource:0}: Error finding container e8d522a5b9f1438a72e758570ecd054f15a3ecff3297bb5921fe229df2ba4beb: Status 404 returned error can't find the container with id e8d522a5b9f1438a72e758570ecd054f15a3ecff3297bb5921fe229df2ba4beb Jun 10 17:32:35.842346 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.842315 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8l6dz" event={"ID":"b15fa61d-010f-4536-9329-1b5b95efc2d4","Type":"ContainerStarted","Data":"6c1e2729bb3618d00d0133d310fbc595e00d5cfdc1664c1ddaca43cfdf2f16cc"} Jun 10 17:32:35.843312 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:35.843289 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6wslj" event={"ID":"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87","Type":"ContainerStarted","Data":"e8d522a5b9f1438a72e758570ecd054f15a3ecff3297bb5921fe229df2ba4beb"} Jun 10 17:32:37.305161 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:37.305063 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:32:37.305593 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:37.305248 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:32:37.305593 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:32:37.305326 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:33:09.305307782 +0000 UTC m=+99.273513801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:32:40.860246 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:40.860162 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8l6dz" event={"ID":"b15fa61d-010f-4536-9329-1b5b95efc2d4","Type":"ContainerStarted","Data":"242ea011219424a13a5393a70852f048d1c55ea2d2362c6012b72c359825e15b"} Jun 10 17:32:40.860603 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:40.860340 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:32:40.861694 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:40.861667 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6wslj" event={"ID":"70f889fb-10f8-4bc7-a0a2-42e0ab79dc87","Type":"ContainerStarted","Data":"e69511946c997f40f2b5eda6e4179f8a075f472291788fc14dbe2ed366d35b66"} Jun 10 17:32:40.874405 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:40.874362 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8l6dz" podStartSLOduration=65.984136533 podStartE2EDuration="1m10.87435001s" totalImagesPullingTime="4.890213477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:32:40.873843144 +0000 UTC m=+70.842049183" watchObservedRunningTime="2026-06-10 17:32:40.87435001 +0000 UTC m=+70.842556112" Jun 10 17:32:40.886252 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:32:40.886206 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6wslj" podStartSLOduration=65.999876115 podStartE2EDuration="1m10.886192783s" totalImagesPullingTime="4.886316668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:32:40.885887378 +0000 UTC m=+70.854093418" watchObservedRunningTime="2026-06-10 17:32:40.886192783 +0000 UTC m=+70.854398822" Jun 10 17:33:09.315816 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:33:09.315680 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:33:09.316299 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:33:09.315840 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:33:09.316299 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:33:09.315910 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:34:13.315893888 +0000 UTC m=+163.284099906 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:33:11.867463 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:33:11.867432 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8l6dz" Jun 10 17:33:39.318805 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:33:39.318747 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:33:39.319317 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:33:39.318910 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 17:33:39.319317 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:33:39.318995 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs podName:d9f00b72-1079-4ead-a63f-5761b40d3166 nodeName:}" failed. No retries permitted until 2026-06-10 17:35:41.31897849 +0000 UTC m=+251.287184509 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs") pod "network-metrics-daemon-f5f66" (UID: "d9f00b72-1079-4ead-a63f-5761b40d3166") : secret "metrics-daemon-secret" not found Jun 10 17:34:08.492780 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:34:08.492728 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2227b" podUID="6f80cc11-56c3-4148-bd3f-17559675eaed" Jun 10 17:34:08.498892 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:34:08.498868 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-f5f66" podUID="d9f00b72-1079-4ead-a63f-5761b40d3166" Jun 10 17:34:09.109550 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:09.109517 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2227b" Jun 10 17:34:13.349802 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:13.349772 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:34:13.350315 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:34:13.349890 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 17:34:13.350315 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:34:13.349998 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls podName:6f80cc11-56c3-4148-bd3f-17559675eaed nodeName:}" failed. No retries permitted until 2026-06-10 17:36:15.349975034 +0000 UTC m=+285.318181057 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls") pod "dns-default-2227b" (UID: "6f80cc11-56c3-4148-bd3f-17559675eaed") : secret "dns-default-metrics-tls" not found Jun 10 17:34:21.464020 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:21.463908 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:34:25.808958 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:25.808886 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" podUID="3944211e-a2f9-4c8e-98a2-689228b4cd9f" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.9:8000/readyz\": dial tcp 10.134.0.9:8000: connect: connection refused" Jun 10 17:34:26.159388 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:26.159310 2531 generic.go:356] "Generic (PLEG): container finished" podID="3944211e-a2f9-4c8e-98a2-689228b4cd9f" containerID="2d5cd1e193b107b5f047ea37a95e11e99e1432cb948da0b15b31724b09b92710" exitCode=1 Jun 10 17:34:26.159388 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:26.159352 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" event={"ID":"3944211e-a2f9-4c8e-98a2-689228b4cd9f","Type":"ContainerDied","Data":"2d5cd1e193b107b5f047ea37a95e11e99e1432cb948da0b15b31724b09b92710"} Jun 10 17:34:26.159668 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:26.159654 2531 scope.go:117] "RemoveContainer" containerID="2d5cd1e193b107b5f047ea37a95e11e99e1432cb948da0b15b31724b09b92710" Jun 10 17:34:27.164824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:27.164790 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" event={"ID":"3944211e-a2f9-4c8e-98a2-689228b4cd9f","Type":"ContainerStarted","Data":"45363f69d0fba7616a76d04db0469e64089536ad4b04dd0bb4471a5828d502b4"} Jun 10 17:34:27.165374 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:27.165082 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:34:27.165655 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:27.165639 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9855bfc7-dr267" Jun 10 17:34:54.862190 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:54.862157 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n8cmd_04b03051-fc01-4dac-a94b-b7ddc921fdf2/dns-node-resolver/0.log" Jun 10 17:34:55.860956 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:55.860914 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vsqlg_111e361e-8521-49d3-9685-9b21257ea3d3/node-ca/0.log" Jun 10 17:34:56.660618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:34:56.660592 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tw5jz_6fc5ad1d-f684-4158-b1e1-bac37539d5f9/serve-healthcheck-canary/0.log" Jun 10 17:35:12.206608 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.206573 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9kznn"] Jun 10 17:35:12.209616 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.209596 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.212469 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.212439 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 10 17:35:12.212586 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.212498 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 10 17:35:12.212586 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.212539 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m95g2\"" Jun 10 17:35:12.212674 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.212587 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 10 17:35:12.212674 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.212544 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 10 17:35:12.220269 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.220244 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9kznn"] Jun 10 17:35:12.351228 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.351201 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.351380 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.351243 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-data-volume\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.351380 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.351289 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f28n\" (UniqueName: \"kubernetes.io/projected/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-kube-api-access-9f28n\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.351380 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.351317 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.351380 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.351344 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-crio-socket\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.451648 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.451623 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f28n\" (UniqueName: \"kubernetes.io/projected/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-kube-api-access-9f28n\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.451744 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.451652 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.451744 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.451671 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-crio-socket\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.451744 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.451706 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.451744 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.451728 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-data-volume\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.451920 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.451786 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-crio-socket\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.452106 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.452089 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-data-volume\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.452411 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.452392 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.454095 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.454075 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.459875 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.459819 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f28n\" (UniqueName: \"kubernetes.io/projected/0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7-kube-api-access-9f28n\") pod \"insights-runtime-extractor-9kznn\" (UID: \"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7\") " pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.518508 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.518489 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9kznn" Jun 10 17:35:12.638610 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:12.638584 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9kznn"] Jun 10 17:35:12.641672 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:35:12.641644 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd8a6cd_4fbb_4c25_a36e_cfcda319b7e7.slice/crio-97fa084c5b15fda3254e16496c94b3e74088bbc1c0e0f04fa9be7d96d1a625e1 WatchSource:0}: Error finding container 97fa084c5b15fda3254e16496c94b3e74088bbc1c0e0f04fa9be7d96d1a625e1: Status 404 returned error can't find the container with id 97fa084c5b15fda3254e16496c94b3e74088bbc1c0e0f04fa9be7d96d1a625e1 Jun 10 17:35:13.294058 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:13.294038 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9kznn" event={"ID":"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7","Type":"ContainerStarted","Data":"0541159d0ed4504f7291b40eaf945bbb27ffaa63327b0b37832d5e27629eebe1"} Jun 10 17:35:13.294321 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:13.294089 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9kznn" event={"ID":"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7","Type":"ContainerStarted","Data":"97fa084c5b15fda3254e16496c94b3e74088bbc1c0e0f04fa9be7d96d1a625e1"} Jun 10 17:35:14.299610 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:14.299565 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9kznn" event={"ID":"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7","Type":"ContainerStarted","Data":"1840ee6e02f2b4e05c3eb341ebed532e249a9fea6bc09c75e417e3c7c249d9c7"} Jun 10 17:35:15.304999 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:15.304960 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9kznn" event={"ID":"0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7","Type":"ContainerStarted","Data":"6c67b52f9cb7e31a9abbeadbf10b09f00762edb284f236e1147ab07c817848b5"} Jun 10 17:35:15.321797 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:15.321736 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9kznn" podStartSLOduration=1.387014111 podStartE2EDuration="3.321709349s" totalImagesPullingTime="1.934695238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 17:35:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:35:15.320504631 +0000 UTC m=+225.288710654" watchObservedRunningTime="2026-06-10 17:35:15.321709349 +0000 UTC m=+225.289915389" Jun 10 17:35:19.351336 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:19.351299 2531 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" podUID="b2fc77d5-50ae-4fba-882d-3c9f43931519" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 10 17:35:29.271681 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.271649 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ttbfl"] Jun 10 17:35:29.274868 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.274848 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.277088 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.277058 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 10 17:35:29.277205 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.277102 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 10 17:35:29.277205 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.277135 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 10 17:35:29.277377 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.277363 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 10 17:35:29.277758 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.277741 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 10 17:35:29.277968 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.277953 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 10 17:35:29.278076 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.278057 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wg8c4\"" Jun 10 17:35:29.351448 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.351409 2531 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" podUID="b2fc77d5-50ae-4fba-882d-3c9f43931519" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 10 17:35:29.371984 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.371961 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-sys\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.372084 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.372011 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-root\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.372084 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.372036 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-wtmp\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.372169 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.372082 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngzq6\" (UniqueName: \"kubernetes.io/projected/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-kube-api-access-ngzq6\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.372169 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.372099 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-tls\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.372169 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.372121 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.372169 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.372153 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.372305 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.372193 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-metrics-client-ca\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.372305 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.372229 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-textfile\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473090 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473060 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-textfile\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473220 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473141 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-sys\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473220 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473195 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-root\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473331 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473223 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-wtmp\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473331 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473256 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-sys\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473331 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473271 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngzq6\" (UniqueName: \"kubernetes.io/projected/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-kube-api-access-ngzq6\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473331 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473298 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-tls\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473331 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473323 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-root\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473331 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473327 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473590 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473356 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473590 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473385 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-textfile\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473590 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473411 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-wtmp\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473590 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.473454 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-metrics-client-ca\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.473590 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:35:29.473457 2531 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 10 17:35:29.473590 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:35:29.473544 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-tls podName:a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2 nodeName:}" failed. No retries permitted until 2026-06-10 17:35:29.973527676 +0000 UTC m=+239.941733713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-tls") pod "node-exporter-ttbfl" (UID: "a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2") : secret "node-exporter-tls" not found Jun 10 17:35:29.474022 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.474000 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.474104 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.474087 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-metrics-client-ca\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.475803 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.475779 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.480914 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.480872 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngzq6\" (UniqueName: \"kubernetes.io/projected/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-kube-api-access-ngzq6\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.977600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.977566 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-tls\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:29.979908 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:29.979883 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2-node-exporter-tls\") pod \"node-exporter-ttbfl\" (UID: \"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2\") " pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:30.184116 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:30.184088 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ttbfl" Jun 10 17:35:30.194313 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:35:30.194287 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a3477b_eaa4_4c6d_9710_f1c7be1e91d2.slice/crio-27c8a9195e9a425078ad2626e9094011c47274213bd6e120ebad6bdfd93cdd0b WatchSource:0}: Error finding container 27c8a9195e9a425078ad2626e9094011c47274213bd6e120ebad6bdfd93cdd0b: Status 404 returned error can't find the container with id 27c8a9195e9a425078ad2626e9094011c47274213bd6e120ebad6bdfd93cdd0b Jun 10 17:35:30.347143 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:30.347112 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ttbfl" event={"ID":"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2","Type":"ContainerStarted","Data":"27c8a9195e9a425078ad2626e9094011c47274213bd6e120ebad6bdfd93cdd0b"} Jun 10 17:35:31.344098 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.344070 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-45hxm"] Jun 10 17:35:31.347059 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.347044 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-45hxm" Jun 10 17:35:31.349241 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.349218 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 10 17:35:31.349241 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.349220 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bwm2t\"" Jun 10 17:35:31.349662 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.349321 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 10 17:35:31.352223 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.352193 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ttbfl" event={"ID":"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2","Type":"ContainerStarted","Data":"dc93a7698cdad10a5d81b2d786946b422f0592870a7d94d2bb7a7a568b0dac61"} Jun 10 17:35:31.356535 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.356497 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-45hxm"] Jun 10 17:35:31.490067 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.490040 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7trqg\" (UniqueName: \"kubernetes.io/projected/ab36f2dd-c32b-40db-ba57-9bda6babeff8-kube-api-access-7trqg\") pod \"downloads-6657875f97-45hxm\" (UID: \"ab36f2dd-c32b-40db-ba57-9bda6babeff8\") " pod="openshift-console/downloads-6657875f97-45hxm" Jun 10 17:35:31.590986 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.590885 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7trqg\" (UniqueName: \"kubernetes.io/projected/ab36f2dd-c32b-40db-ba57-9bda6babeff8-kube-api-access-7trqg\") pod \"downloads-6657875f97-45hxm\" (UID: \"ab36f2dd-c32b-40db-ba57-9bda6babeff8\") " pod="openshift-console/downloads-6657875f97-45hxm" Jun 10 17:35:31.602237 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.602209 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7trqg\" (UniqueName: \"kubernetes.io/projected/ab36f2dd-c32b-40db-ba57-9bda6babeff8-kube-api-access-7trqg\") pod \"downloads-6657875f97-45hxm\" (UID: \"ab36f2dd-c32b-40db-ba57-9bda6babeff8\") " pod="openshift-console/downloads-6657875f97-45hxm" Jun 10 17:35:31.657347 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.657319 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-45hxm" Jun 10 17:35:31.776990 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:31.776959 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-45hxm"] Jun 10 17:35:31.779833 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:35:31.779807 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab36f2dd_c32b_40db_ba57_9bda6babeff8.slice/crio-f881f7354e790f93a75791c5f85d966c6f647dc66cb5ceb14951a102c5847886 WatchSource:0}: Error finding container f881f7354e790f93a75791c5f85d966c6f647dc66cb5ceb14951a102c5847886: Status 404 returned error can't find the container with id f881f7354e790f93a75791c5f85d966c6f647dc66cb5ceb14951a102c5847886 Jun 10 17:35:32.357212 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:32.357178 2531 generic.go:356] "Generic (PLEG): container finished" podID="a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2" containerID="dc93a7698cdad10a5d81b2d786946b422f0592870a7d94d2bb7a7a568b0dac61" exitCode=0 Jun 10 17:35:32.357643 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:32.357273 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ttbfl" event={"ID":"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2","Type":"ContainerDied","Data":"dc93a7698cdad10a5d81b2d786946b422f0592870a7d94d2bb7a7a568b0dac61"} Jun 10 17:35:32.358409 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:32.358389 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-45hxm" event={"ID":"ab36f2dd-c32b-40db-ba57-9bda6babeff8","Type":"ContainerStarted","Data":"f881f7354e790f93a75791c5f85d966c6f647dc66cb5ceb14951a102c5847886"} Jun 10 17:35:33.365298 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:33.365260 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ttbfl" event={"ID":"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2","Type":"ContainerStarted","Data":"ab1b9c8dc600b586d70a3d74bc1424b535680848253b1d51b22d58c7174ae990"} Jun 10 17:35:33.365649 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:33.365326 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ttbfl" event={"ID":"a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2","Type":"ContainerStarted","Data":"385a4cdaf29e3f770d474f2c2afbe163b9c6140b0176e28eff6baf7fcbef0cd8"} Jun 10 17:35:33.383196 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:33.383158 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ttbfl" podStartSLOduration=3.245409843 podStartE2EDuration="4.383145706s" totalImagesPullingTime="1.06429396s" totalInitContainerRuntime="73.441903ms" isStatefulPod=true podCreationTimestamp="2026-06-10 17:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:35:33.381569502 +0000 UTC m=+243.349775540" watchObservedRunningTime="2026-06-10 17:35:33.383145706 +0000 UTC m=+243.351351790" Jun 10 17:35:34.023437 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.023402 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-nh54r"] Jun 10 17:35:34.026330 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.026316 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:34.028296 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.028275 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 10 17:35:34.028296 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.028290 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-l5h94\"" Jun 10 17:35:34.033869 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.033850 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-nh54r"] Jun 10 17:35:34.212589 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.212556 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3ddba959-6c32-43d7-9db4-40730e43f44d-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-nh54r\" (UID: \"3ddba959-6c32-43d7-9db4-40730e43f44d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:34.212589 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.212593 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-459bp\" (UniqueName: \"kubernetes.io/projected/3ddba959-6c32-43d7-9db4-40730e43f44d-kube-api-access-459bp\") pod \"monitoring-plugin-76b68cc874-nh54r\" (UID: \"3ddba959-6c32-43d7-9db4-40730e43f44d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:34.314080 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.314006 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3ddba959-6c32-43d7-9db4-40730e43f44d-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-nh54r\" (UID: \"3ddba959-6c32-43d7-9db4-40730e43f44d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:34.314080 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.314043 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-459bp\" (UniqueName: \"kubernetes.io/projected/3ddba959-6c32-43d7-9db4-40730e43f44d-kube-api-access-459bp\") pod \"monitoring-plugin-76b68cc874-nh54r\" (UID: \"3ddba959-6c32-43d7-9db4-40730e43f44d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:34.314257 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:35:34.314161 2531 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 10 17:35:34.314257 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:35:34.314230 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ddba959-6c32-43d7-9db4-40730e43f44d-monitoring-plugin-cert podName:3ddba959-6c32-43d7-9db4-40730e43f44d nodeName:}" failed. No retries permitted until 2026-06-10 17:35:34.814212424 +0000 UTC m=+244.782418444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3ddba959-6c32-43d7-9db4-40730e43f44d-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-nh54r" (UID: "3ddba959-6c32-43d7-9db4-40730e43f44d") : secret "monitoring-plugin-cert" not found Jun 10 17:35:34.324727 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.324698 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-459bp\" (UniqueName: \"kubernetes.io/projected/3ddba959-6c32-43d7-9db4-40730e43f44d-kube-api-access-459bp\") pod \"monitoring-plugin-76b68cc874-nh54r\" (UID: \"3ddba959-6c32-43d7-9db4-40730e43f44d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:34.428265 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.428226 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-549b58c766-hrxj2"] Jun 10 17:35:34.431697 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.431666 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.433704 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.433668 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 10 17:35:34.433704 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.433675 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 10 17:35:34.433903 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.433863 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 10 17:35:34.434021 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.433924 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 10 17:35:34.434162 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.434140 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 10 17:35:34.434245 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.434200 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-42bqm\"" Jun 10 17:35:34.438161 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.438141 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 10 17:35:34.444315 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.444283 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-549b58c766-hrxj2"] Jun 10 17:35:34.616749 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.616664 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-serving-certs-ca-bundle\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.616749 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.616710 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.616989 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.616754 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-federate-client-tls\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.616989 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.616817 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl8s7\" (UniqueName: \"kubernetes.io/projected/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-kube-api-access-tl8s7\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.616989 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.616875 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-secret-telemeter-client\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.616989 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.616918 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.616989 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.616957 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-telemeter-client-tls\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.617244 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.617018 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-metrics-client-ca\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.717704 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.717666 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-metrics-client-ca\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.717704 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.717705 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-serving-certs-ca-bundle\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.717888 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.717727 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.717888 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.717765 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-federate-client-tls\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.717888 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.717785 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl8s7\" (UniqueName: \"kubernetes.io/projected/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-kube-api-access-tl8s7\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.717888 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.717811 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-secret-telemeter-client\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.717888 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.717845 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.717888 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.717871 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-telemeter-client-tls\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.718681 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.718522 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-serving-certs-ca-bundle\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.718681 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.718527 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-metrics-client-ca\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.718681 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.718615 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.720413 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.720393 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-federate-client-tls\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.720592 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.720571 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.720839 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.720825 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-secret-telemeter-client\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.721011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.720993 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-telemeter-client-tls\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.725373 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.725349 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl8s7\" (UniqueName: \"kubernetes.io/projected/eb94fafe-a14f-4db9-9e02-91572e0a8f9a-kube-api-access-tl8s7\") pod \"telemeter-client-549b58c766-hrxj2\" (UID: \"eb94fafe-a14f-4db9-9e02-91572e0a8f9a\") " pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.743179 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.743155 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" Jun 10 17:35:34.818378 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.818322 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3ddba959-6c32-43d7-9db4-40730e43f44d-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-nh54r\" (UID: \"3ddba959-6c32-43d7-9db4-40730e43f44d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:34.820978 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.820902 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3ddba959-6c32-43d7-9db4-40730e43f44d-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-nh54r\" (UID: \"3ddba959-6c32-43d7-9db4-40730e43f44d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:34.885314 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.885237 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-549b58c766-hrxj2"] Jun 10 17:35:34.888732 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:35:34.888703 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb94fafe_a14f_4db9_9e02_91572e0a8f9a.slice/crio-b25fef7e32a8e865709accec614119f964524433f671d1e9ff3252f0b3f3c243 WatchSource:0}: Error finding container b25fef7e32a8e865709accec614119f964524433f671d1e9ff3252f0b3f3c243: Status 404 returned error can't find the container with id b25fef7e32a8e865709accec614119f964524433f671d1e9ff3252f0b3f3c243 Jun 10 17:35:34.934492 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:34.934462 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:35.074721 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:35.074691 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-nh54r"] Jun 10 17:35:35.078360 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:35:35.078330 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ddba959_6c32_43d7_9db4_40730e43f44d.slice/crio-034c6c46ba32f3eea9ae096bb109c75f81af4cc9e98713d3e27a52ac9972292b WatchSource:0}: Error finding container 034c6c46ba32f3eea9ae096bb109c75f81af4cc9e98713d3e27a52ac9972292b: Status 404 returned error can't find the container with id 034c6c46ba32f3eea9ae096bb109c75f81af4cc9e98713d3e27a52ac9972292b Jun 10 17:35:35.372815 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:35.372771 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" event={"ID":"eb94fafe-a14f-4db9-9e02-91572e0a8f9a","Type":"ContainerStarted","Data":"b25fef7e32a8e865709accec614119f964524433f671d1e9ff3252f0b3f3c243"} Jun 10 17:35:35.374048 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:35.374017 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" event={"ID":"3ddba959-6c32-43d7-9db4-40730e43f44d","Type":"ContainerStarted","Data":"034c6c46ba32f3eea9ae096bb109c75f81af4cc9e98713d3e27a52ac9972292b"} Jun 10 17:35:37.383501 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:37.383432 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" event={"ID":"eb94fafe-a14f-4db9-9e02-91572e0a8f9a","Type":"ContainerStarted","Data":"b98ed37fd904772943754fe73244ceb20890b4dbcdc600a5505d8d7ee2ab3e3f"} Jun 10 17:35:37.385097 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:37.385068 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" event={"ID":"3ddba959-6c32-43d7-9db4-40730e43f44d","Type":"ContainerStarted","Data":"0aad61ec09d6c1d51f025b803401169fd4d259d0bd4c82bffacdfd108c27cb3e"} Jun 10 17:35:37.385355 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:37.385323 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:37.391327 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:37.391302 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" Jun 10 17:35:37.399666 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:37.399629 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nh54r" podStartSLOduration=1.23473527 podStartE2EDuration="3.399617113s" totalImagesPullingTime="2.164881843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:35:37.398420467 +0000 UTC m=+247.366626509" watchObservedRunningTime="2026-06-10 17:35:37.399617113 +0000 UTC m=+247.367823152" Jun 10 17:35:38.391650 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:38.391601 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" event={"ID":"eb94fafe-a14f-4db9-9e02-91572e0a8f9a","Type":"ContainerStarted","Data":"e0f90cf66228d315ab743701ef66def654668198a289ef59070660e1a18501f8"} Jun 10 17:35:38.392086 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:38.391687 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" event={"ID":"eb94fafe-a14f-4db9-9e02-91572e0a8f9a","Type":"ContainerStarted","Data":"67885a5585933c76969ba46cf22b7c5842eed6a18d4b1613fe06709b7b8b1cb2"} Jun 10 17:35:38.411697 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:38.411649 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-549b58c766-hrxj2" podStartSLOduration=1.169846575 podStartE2EDuration="4.411632084s" totalImagesPullingTime="3.241785509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:35:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:35:38.409984354 +0000 UTC m=+248.378190394" watchObservedRunningTime="2026-06-10 17:35:38.411632084 +0000 UTC m=+248.379838125" Jun 10 17:35:39.351522 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:39.351478 2531 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" podUID="b2fc77d5-50ae-4fba-882d-3c9f43931519" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 10 17:35:39.351695 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:39.351562 2531 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" Jun 10 17:35:39.352175 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:39.352133 2531 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"04533245ae762a796cc951566b951010d46edc5e25069c2aa01c5da2fd07491a"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 10 17:35:39.352303 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:39.352202 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" podUID="b2fc77d5-50ae-4fba-882d-3c9f43931519" containerName="service-proxy" containerID="cri-o://04533245ae762a796cc951566b951010d46edc5e25069c2aa01c5da2fd07491a" gracePeriod=30 Jun 10 17:35:40.402221 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.402183 2531 generic.go:356] "Generic (PLEG): container finished" podID="b2fc77d5-50ae-4fba-882d-3c9f43931519" containerID="04533245ae762a796cc951566b951010d46edc5e25069c2aa01c5da2fd07491a" exitCode=2 Jun 10 17:35:40.402639 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.402248 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" event={"ID":"b2fc77d5-50ae-4fba-882d-3c9f43931519","Type":"ContainerDied","Data":"04533245ae762a796cc951566b951010d46edc5e25069c2aa01c5da2fd07491a"} Jun 10 17:35:40.402639 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.402290 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b68464d89-d5g6x" event={"ID":"b2fc77d5-50ae-4fba-882d-3c9f43931519","Type":"ContainerStarted","Data":"2da1bc4a1ab46d657c25d84766babe597d4790e1d54cc799d29c9b0f1b1492a5"} Jun 10 17:35:40.457282 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.457234 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b55f4c47c-94g2h"] Jun 10 17:35:40.459931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.459907 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.462130 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.462106 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 10 17:35:40.462253 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.462171 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l4wmp\"" Jun 10 17:35:40.462253 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.462242 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 10 17:35:40.462739 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.462570 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 10 17:35:40.462739 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.462612 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 10 17:35:40.462739 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.462685 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 10 17:35:40.468539 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.468515 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-console-config\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.468637 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.468581 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-service-ca\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.468637 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.468618 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-serving-cert\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.468746 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.468642 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-oauth-serving-cert\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.468746 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.468699 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d7bx\" (UniqueName: \"kubernetes.io/projected/a4eaa093-1dbd-47a5-9168-0873332a0539-kube-api-access-5d7bx\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.468854 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.468838 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-oauth-config\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.474301 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.474273 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b55f4c47c-94g2h"] Jun 10 17:35:40.569874 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.569838 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-service-ca\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.570067 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.569891 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-serving-cert\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.570067 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.569917 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-oauth-serving-cert\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.570067 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.569977 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d7bx\" (UniqueName: \"kubernetes.io/projected/a4eaa093-1dbd-47a5-9168-0873332a0539-kube-api-access-5d7bx\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.570067 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.570055 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-oauth-config\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.570276 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.570087 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-console-config\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.570661 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.570633 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-service-ca\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.570923 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.570693 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-oauth-serving-cert\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.570923 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.570734 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-console-config\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.573270 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.573246 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-oauth-config\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.573372 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.573282 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-serving-cert\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.578362 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.578341 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d7bx\" (UniqueName: \"kubernetes.io/projected/a4eaa093-1dbd-47a5-9168-0873332a0539-kube-api-access-5d7bx\") pod \"console-6b55f4c47c-94g2h\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.771560 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.771529 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:35:40.900851 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:40.900816 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b55f4c47c-94g2h"] Jun 10 17:35:40.905189 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:35:40.905163 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4eaa093_1dbd_47a5_9168_0873332a0539.slice/crio-8781e6ac4caf9c5cabf81e0b90336fb1cc4be26b78b2e148240c0d9a049c24a7 WatchSource:0}: Error finding container 8781e6ac4caf9c5cabf81e0b90336fb1cc4be26b78b2e148240c0d9a049c24a7: Status 404 returned error can't find the container with id 8781e6ac4caf9c5cabf81e0b90336fb1cc4be26b78b2e148240c0d9a049c24a7 Jun 10 17:35:41.376535 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:41.376496 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:35:41.379017 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:41.378991 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f00b72-1079-4ead-a63f-5761b40d3166-metrics-certs\") pod \"network-metrics-daemon-f5f66\" (UID: \"d9f00b72-1079-4ead-a63f-5761b40d3166\") " pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:35:41.406600 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:41.406573 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b55f4c47c-94g2h" event={"ID":"a4eaa093-1dbd-47a5-9168-0873332a0539","Type":"ContainerStarted","Data":"8781e6ac4caf9c5cabf81e0b90336fb1cc4be26b78b2e148240c0d9a049c24a7"} Jun 10 17:35:41.566995 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:41.566958 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d9nks\"" Jun 10 17:35:41.575813 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:41.575783 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5f66" Jun 10 17:35:41.744035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:41.744001 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f5f66"] Jun 10 17:35:41.749025 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:35:41.748995 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f00b72_1079_4ead_a63f_5761b40d3166.slice/crio-e815cacbf31b7b061f0497b139f1678499e386b85e363edb528449fd6cb84c1e WatchSource:0}: Error finding container e815cacbf31b7b061f0497b139f1678499e386b85e363edb528449fd6cb84c1e: Status 404 returned error can't find the container with id e815cacbf31b7b061f0497b139f1678499e386b85e363edb528449fd6cb84c1e Jun 10 17:35:42.412949 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:42.412897 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5f66" event={"ID":"d9f00b72-1079-4ead-a63f-5761b40d3166","Type":"ContainerStarted","Data":"e815cacbf31b7b061f0497b139f1678499e386b85e363edb528449fd6cb84c1e"} Jun 10 17:35:43.419201 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:43.419167 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5f66" event={"ID":"d9f00b72-1079-4ead-a63f-5761b40d3166","Type":"ContainerStarted","Data":"4ed6b3026067d7337e1eee138d16c53636e96b328dc2fcc583d003e19fc7e8b6"} Jun 10 17:35:43.419651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:43.419228 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5f66" event={"ID":"d9f00b72-1079-4ead-a63f-5761b40d3166","Type":"ContainerStarted","Data":"bfa7d6f98d2f1cea5e6a4b595a538f57f949832d51d9deeb33ca5a2f269e213e"} Jun 10 17:35:43.434748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:43.434692 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f5f66" podStartSLOduration=252.508760446 podStartE2EDuration="4m13.434673249s" totalImagesPullingTime="925.912803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:35:43.433610739 +0000 UTC m=+253.401816780" watchObservedRunningTime="2026-06-10 17:35:43.434673249 +0000 UTC m=+253.402879289" Jun 10 17:35:48.237163 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.237075 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c7b777ddd-hcggj"] Jun 10 17:35:48.240588 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.240562 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.248002 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.247976 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 10 17:35:48.250628 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.250481 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7b777ddd-hcggj"] Jun 10 17:35:48.339856 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.339815 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-oauth-serving-cert\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.339856 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.339859 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7kdd\" (UniqueName: \"kubernetes.io/projected/d73705b1-d9d1-45c6-af71-edc45e54e6cb-kube-api-access-n7kdd\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.340110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.339879 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-trusted-ca-bundle\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.340110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.339922 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-oauth-config\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.340110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.340015 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-serving-cert\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.340110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.340062 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-service-ca\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.340110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.340084 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-config\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.440502 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.440472 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-service-ca\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.440502 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.440506 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-config\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.440748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.440575 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-oauth-serving-cert\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.440748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.440601 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7kdd\" (UniqueName: \"kubernetes.io/projected/d73705b1-d9d1-45c6-af71-edc45e54e6cb-kube-api-access-n7kdd\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.440748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.440628 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-trusted-ca-bundle\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.440748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.440661 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-oauth-config\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.440748 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.440690 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-serving-cert\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.441406 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.441375 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-service-ca\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.441701 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.441385 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-config\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.442022 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.441997 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-trusted-ca-bundle\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.442125 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.442017 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-oauth-serving-cert\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.443725 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.443690 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-oauth-config\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.444065 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.444046 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-serving-cert\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.448592 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.448569 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7kdd\" (UniqueName: \"kubernetes.io/projected/d73705b1-d9d1-45c6-af71-edc45e54e6cb-kube-api-access-n7kdd\") pod \"console-6c7b777ddd-hcggj\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:48.551577 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:48.551546 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:51.109828 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.109779 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7b777ddd-hcggj"] Jun 10 17:35:51.112307 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:35:51.112279 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73705b1_d9d1_45c6_af71_edc45e54e6cb.slice/crio-16797055cdeb4fbf24eca0870164f86b1e764d3a6bd32328e7b221285d1711af WatchSource:0}: Error finding container 16797055cdeb4fbf24eca0870164f86b1e764d3a6bd32328e7b221285d1711af: Status 404 returned error can't find the container with id 16797055cdeb4fbf24eca0870164f86b1e764d3a6bd32328e7b221285d1711af Jun 10 17:35:51.453109 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.453065 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b55f4c47c-94g2h" event={"ID":"a4eaa093-1dbd-47a5-9168-0873332a0539","Type":"ContainerStarted","Data":"53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc"} Jun 10 17:35:51.455105 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.455077 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7b777ddd-hcggj" event={"ID":"d73705b1-d9d1-45c6-af71-edc45e54e6cb","Type":"ContainerStarted","Data":"e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d"} Jun 10 17:35:51.455252 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.455122 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7b777ddd-hcggj" event={"ID":"d73705b1-d9d1-45c6-af71-edc45e54e6cb","Type":"ContainerStarted","Data":"16797055cdeb4fbf24eca0870164f86b1e764d3a6bd32328e7b221285d1711af"} Jun 10 17:35:51.456928 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.456909 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-45hxm" event={"ID":"ab36f2dd-c32b-40db-ba57-9bda6babeff8","Type":"ContainerStarted","Data":"096af9888e0d02ab514e08338f1c9a21fc71d66f10da5199cd974fb1cacc6346"} Jun 10 17:35:51.457167 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.457142 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-45hxm" Jun 10 17:35:51.468247 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.468187 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b55f4c47c-94g2h" podStartSLOduration=1.393522911 podStartE2EDuration="11.468172006s" totalImagesPullingTime="10.074649095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:35:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:35:51.46808812 +0000 UTC m=+261.436294170" watchObservedRunningTime="2026-06-10 17:35:51.468172006 +0000 UTC m=+261.436378046" Jun 10 17:35:51.472277 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.472254 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-45hxm" Jun 10 17:35:51.483719 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.483679 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-45hxm" podStartSLOduration=1.222559057 podStartE2EDuration="20.483663998s" totalImagesPullingTime="19.261104941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:35:51.481855706 +0000 UTC m=+261.450061746" watchObservedRunningTime="2026-06-10 17:35:51.483663998 +0000 UTC m=+261.451870038" Jun 10 17:35:51.496961 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:51.496899 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c7b777ddd-hcggj" podStartSLOduration=3.496883261 podStartE2EDuration="3.496883261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:35:51.495995571 +0000 UTC m=+261.464201611" watchObservedRunningTime="2026-06-10 17:35:51.496883261 +0000 UTC m=+261.465089300" Jun 10 17:35:58.552197 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:58.552156 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:58.552727 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:58.552235 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:35:58.553764 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:58.553738 2531 patch_prober.go:28] interesting pod/console-6c7b777ddd-hcggj container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.15:8443/health\": dial tcp 10.134.0.15:8443: connect: connection refused" start-of-body= Jun 10 17:35:58.553900 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:35:58.553790 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6c7b777ddd-hcggj" podUID="d73705b1-d9d1-45c6-af71-edc45e54e6cb" containerName="console" probeResult="failure" output="Get \"https://10.134.0.15:8443/health\": dial tcp 10.134.0.15:8443: connect: connection refused" Jun 10 17:36:00.772507 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:00.772475 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:36:00.773064 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:00.772527 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:36:00.773900 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:00.773878 2531 patch_prober.go:28] interesting pod/console-6b55f4c47c-94g2h container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.14:8443/health\": dial tcp 10.134.0.14:8443: connect: connection refused" start-of-body= Jun 10 17:36:00.774008 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:00.773917 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6b55f4c47c-94g2h" podUID="a4eaa093-1dbd-47a5-9168-0873332a0539" containerName="console" probeResult="failure" output="Get \"https://10.134.0.14:8443/health\": dial tcp 10.134.0.14:8443: connect: connection refused" Jun 10 17:36:08.552192 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:08.552156 2531 patch_prober.go:28] interesting pod/console-6c7b777ddd-hcggj container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.15:8443/health\": dial tcp 10.134.0.15:8443: connect: connection refused" start-of-body= Jun 10 17:36:08.552628 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:08.552205 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6c7b777ddd-hcggj" podUID="d73705b1-d9d1-45c6-af71-edc45e54e6cb" containerName="console" probeResult="failure" output="Get \"https://10.134.0.15:8443/health\": dial tcp 10.134.0.15:8443: connect: connection refused" Jun 10 17:36:10.772853 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:10.772821 2531 patch_prober.go:28] interesting pod/console-6b55f4c47c-94g2h container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.14:8443/health\": dial tcp 10.134.0.14:8443: connect: connection refused" start-of-body= Jun 10 17:36:10.773230 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:10.772870 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6b55f4c47c-94g2h" podUID="a4eaa093-1dbd-47a5-9168-0873332a0539" containerName="console" probeResult="failure" output="Get \"https://10.134.0.14:8443/health\": dial tcp 10.134.0.14:8443: connect: connection refused" Jun 10 17:36:12.110683 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:36:12.110642 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2227b" podUID="6f80cc11-56c3-4148-bd3f-17559675eaed" Jun 10 17:36:12.531788 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:12.531751 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2227b" Jun 10 17:36:15.371257 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:15.371219 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:36:15.373578 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:15.373559 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6f80cc11-56c3-4148-bd3f-17559675eaed-metrics-tls\") pod \"dns-default-2227b\" (UID: \"6f80cc11-56c3-4148-bd3f-17559675eaed\") " pod="openshift-dns/dns-default-2227b" Jun 10 17:36:15.535286 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:15.535253 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dqnwj\"" Jun 10 17:36:15.544233 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:15.544212 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2227b" Jun 10 17:36:15.672694 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:15.672668 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2227b"] Jun 10 17:36:15.675417 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:36:15.675393 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f80cc11_56c3_4148_bd3f_17559675eaed.slice/crio-9e2b1f70c1a70c432c02f603c097ce925ed60204876fc567e10e5c806b2c7ae8 WatchSource:0}: Error finding container 9e2b1f70c1a70c432c02f603c097ce925ed60204876fc567e10e5c806b2c7ae8: Status 404 returned error can't find the container with id 9e2b1f70c1a70c432c02f603c097ce925ed60204876fc567e10e5c806b2c7ae8 Jun 10 17:36:16.547035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:16.546993 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2227b" event={"ID":"6f80cc11-56c3-4148-bd3f-17559675eaed","Type":"ContainerStarted","Data":"9e2b1f70c1a70c432c02f603c097ce925ed60204876fc567e10e5c806b2c7ae8"} Jun 10 17:36:18.556266 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:18.556235 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:36:18.556681 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:18.556558 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2227b" event={"ID":"6f80cc11-56c3-4148-bd3f-17559675eaed","Type":"ContainerStarted","Data":"14f72a50b8648b7a4a7dda14b58ec342feb924aa8f8138d369b48be30b370974"} Jun 10 17:36:18.556681 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:18.556612 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2227b" event={"ID":"6f80cc11-56c3-4148-bd3f-17559675eaed","Type":"ContainerStarted","Data":"b60900fee2f8c7865d402f900876dcfad0820da786ef3728f5f5dc535604f818"} Jun 10 17:36:18.556777 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:18.556746 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2227b" Jun 10 17:36:18.560211 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:18.560190 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:36:18.584866 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:18.584734 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2227b" podStartSLOduration=251.721031511 podStartE2EDuration="4m13.584716144s" totalImagesPullingTime="1.863684633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:36:18.584093892 +0000 UTC m=+288.552299954" watchObservedRunningTime="2026-06-10 17:36:18.584716144 +0000 UTC m=+288.552922189" Jun 10 17:36:18.623560 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:18.623530 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b55f4c47c-94g2h"] Jun 10 17:36:28.561531 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:28.561503 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2227b" Jun 10 17:36:30.406328 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:30.406297 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:36:30.411124 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:30.411100 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:36:30.414140 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:30.414118 2531 kubelet.go:1648] "Image garbage collection succeeded" Jun 10 17:36:43.643138 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.643072 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b55f4c47c-94g2h" podUID="a4eaa093-1dbd-47a5-9168-0873332a0539" containerName="console" containerID="cri-o://53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc" gracePeriod=15 Jun 10 17:36:43.879377 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.879355 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:36:43.977825 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.977744 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-console-config\") pod \"a4eaa093-1dbd-47a5-9168-0873332a0539\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " Jun 10 17:36:43.977825 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.977778 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-service-ca\") pod \"a4eaa093-1dbd-47a5-9168-0873332a0539\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " Jun 10 17:36:43.977825 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.977813 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-serving-cert\") pod \"a4eaa093-1dbd-47a5-9168-0873332a0539\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " Jun 10 17:36:43.978116 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.977866 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d7bx\" (UniqueName: \"kubernetes.io/projected/a4eaa093-1dbd-47a5-9168-0873332a0539-kube-api-access-5d7bx\") pod \"a4eaa093-1dbd-47a5-9168-0873332a0539\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " Jun 10 17:36:43.978116 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.977884 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-oauth-config\") pod \"a4eaa093-1dbd-47a5-9168-0873332a0539\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " Jun 10 17:36:43.978116 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.977916 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-oauth-serving-cert\") pod \"a4eaa093-1dbd-47a5-9168-0873332a0539\" (UID: \"a4eaa093-1dbd-47a5-9168-0873332a0539\") " Jun 10 17:36:43.978308 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.978275 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-service-ca" (OuterVolumeSpecName: "service-ca") pod "a4eaa093-1dbd-47a5-9168-0873332a0539" (UID: "a4eaa093-1dbd-47a5-9168-0873332a0539"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:36:43.978308 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.978190 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-console-config" (OuterVolumeSpecName: "console-config") pod "a4eaa093-1dbd-47a5-9168-0873332a0539" (UID: "a4eaa093-1dbd-47a5-9168-0873332a0539"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:36:43.978459 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.978386 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a4eaa093-1dbd-47a5-9168-0873332a0539" (UID: "a4eaa093-1dbd-47a5-9168-0873332a0539"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:36:43.980196 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.980166 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4eaa093-1dbd-47a5-9168-0873332a0539-kube-api-access-5d7bx" (OuterVolumeSpecName: "kube-api-access-5d7bx") pod "a4eaa093-1dbd-47a5-9168-0873332a0539" (UID: "a4eaa093-1dbd-47a5-9168-0873332a0539"). InnerVolumeSpecName "kube-api-access-5d7bx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:36:43.980535 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.980515 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a4eaa093-1dbd-47a5-9168-0873332a0539" (UID: "a4eaa093-1dbd-47a5-9168-0873332a0539"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:36:43.980628 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:43.980503 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a4eaa093-1dbd-47a5-9168-0873332a0539" (UID: "a4eaa093-1dbd-47a5-9168-0873332a0539"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:36:44.078428 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.078401 2531 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:36:44.078428 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.078426 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d7bx\" (UniqueName: \"kubernetes.io/projected/a4eaa093-1dbd-47a5-9168-0873332a0539-kube-api-access-5d7bx\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:36:44.078579 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.078435 2531 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4eaa093-1dbd-47a5-9168-0873332a0539-console-oauth-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:36:44.078579 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.078444 2531 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-oauth-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:36:44.078579 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.078454 2531 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-console-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:36:44.078579 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.078462 2531 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4eaa093-1dbd-47a5-9168-0873332a0539-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:36:44.648773 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.648741 2531 generic.go:356] "Generic (PLEG): container finished" podID="a4eaa093-1dbd-47a5-9168-0873332a0539" containerID="53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc" exitCode=0 Jun 10 17:36:44.649227 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.648782 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b55f4c47c-94g2h" Jun 10 17:36:44.649227 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.648799 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b55f4c47c-94g2h" event={"ID":"a4eaa093-1dbd-47a5-9168-0873332a0539","Type":"ContainerDied","Data":"53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc"} Jun 10 17:36:44.649227 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.648848 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b55f4c47c-94g2h" event={"ID":"a4eaa093-1dbd-47a5-9168-0873332a0539","Type":"ContainerDied","Data":"8781e6ac4caf9c5cabf81e0b90336fb1cc4be26b78b2e148240c0d9a049c24a7"} Jun 10 17:36:44.649227 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.648868 2531 scope.go:117] "RemoveContainer" containerID="53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc" Jun 10 17:36:44.656924 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.656909 2531 scope.go:117] "RemoveContainer" containerID="53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc" Jun 10 17:36:44.657205 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:36:44.657185 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc\": container with ID starting with 53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc not found: ID does not exist" containerID="53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc" Jun 10 17:36:44.657265 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.657217 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc"} err="failed to get container status \"53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc\": rpc error: code = NotFound desc = could not find container \"53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc\": container with ID starting with 53f24c8246652df521cdd1ef1022a5ee166afd108d1103ec52c3e23c9a61acdc not found: ID does not exist" Jun 10 17:36:44.662765 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.662744 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b55f4c47c-94g2h"] Jun 10 17:36:44.666124 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:44.666105 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b55f4c47c-94g2h"] Jun 10 17:36:46.468305 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:46.468267 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4eaa093-1dbd-47a5-9168-0873332a0539" path="/var/lib/kubelet/pods/a4eaa093-1dbd-47a5-9168-0873332a0539/volumes" Jun 10 17:36:53.734663 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.734627 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c784d7ffb-smwcx"] Jun 10 17:36:53.735045 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.734886 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4eaa093-1dbd-47a5-9168-0873332a0539" containerName="console" Jun 10 17:36:53.735045 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.734898 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4eaa093-1dbd-47a5-9168-0873332a0539" containerName="console" Jun 10 17:36:53.735045 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.734963 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4eaa093-1dbd-47a5-9168-0873332a0539" containerName="console" Jun 10 17:36:53.736913 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.736895 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.747998 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.747974 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c784d7ffb-smwcx"] Jun 10 17:36:53.850581 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.850549 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-service-ca\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.850751 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.850586 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-oauth-config\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.850751 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.850618 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-trusted-ca-bundle\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.850751 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.850693 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-serving-cert\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.850899 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.850789 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-oauth-serving-cert\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.850899 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.850810 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drmzt\" (UniqueName: \"kubernetes.io/projected/97606067-13ac-4e6a-b5cd-9589857e9003-kube-api-access-drmzt\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.850899 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.850871 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-console-config\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.952012 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.951967 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-oauth-config\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.952012 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.952014 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-trusted-ca-bundle\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.952228 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.952041 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-serving-cert\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.952228 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.952102 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-oauth-serving-cert\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.952228 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.952129 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drmzt\" (UniqueName: \"kubernetes.io/projected/97606067-13ac-4e6a-b5cd-9589857e9003-kube-api-access-drmzt\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.952228 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.952168 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-console-config\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.952385 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.952257 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-service-ca\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.957146 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.953495 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-trusted-ca-bundle\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.957146 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.953407 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-console-config\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.957146 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.953713 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-oauth-serving-cert\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.957146 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.953724 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-service-ca\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.957558 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.957533 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-oauth-config\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.957685 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.957667 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-serving-cert\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:53.960517 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:53.960494 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drmzt\" (UniqueName: \"kubernetes.io/projected/97606067-13ac-4e6a-b5cd-9589857e9003-kube-api-access-drmzt\") pod \"console-5c784d7ffb-smwcx\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:54.045640 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:54.045609 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:36:54.175302 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:54.175166 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c784d7ffb-smwcx"] Jun 10 17:36:54.177837 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:36:54.177811 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97606067_13ac_4e6a_b5cd_9589857e9003.slice/crio-e1dfe7cb7bd6c5ba365bc131f3ebfa18f4d72e2a0cb658fbabdd31ce7cd4e1e4 WatchSource:0}: Error finding container e1dfe7cb7bd6c5ba365bc131f3ebfa18f4d72e2a0cb658fbabdd31ce7cd4e1e4: Status 404 returned error can't find the container with id e1dfe7cb7bd6c5ba365bc131f3ebfa18f4d72e2a0cb658fbabdd31ce7cd4e1e4 Jun 10 17:36:54.684247 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:54.684210 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c784d7ffb-smwcx" event={"ID":"97606067-13ac-4e6a-b5cd-9589857e9003","Type":"ContainerStarted","Data":"be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b"} Jun 10 17:36:54.684436 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:54.684258 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c784d7ffb-smwcx" event={"ID":"97606067-13ac-4e6a-b5cd-9589857e9003","Type":"ContainerStarted","Data":"e1dfe7cb7bd6c5ba365bc131f3ebfa18f4d72e2a0cb658fbabdd31ce7cd4e1e4"} Jun 10 17:36:54.700991 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:36:54.700921 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c784d7ffb-smwcx" podStartSLOduration=1.700908818 podStartE2EDuration="1.700908818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:36:54.69931096 +0000 UTC m=+324.667517000" watchObservedRunningTime="2026-06-10 17:36:54.700908818 +0000 UTC m=+324.669114856" Jun 10 17:37:04.045747 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:04.045710 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:37:04.046325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:04.045772 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:37:04.050472 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:04.050451 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:37:04.723340 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:04.723311 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:37:04.765110 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:04.765073 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7b777ddd-hcggj"] Jun 10 17:37:29.785128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:29.785022 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6c7b777ddd-hcggj" podUID="d73705b1-d9d1-45c6-af71-edc45e54e6cb" containerName="console" containerID="cri-o://e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d" gracePeriod=15 Jun 10 17:37:30.022640 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.022612 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:37:30.126707 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.126631 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-trusted-ca-bundle\") pod \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " Jun 10 17:37:30.126707 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.126665 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-config\") pod \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " Jun 10 17:37:30.126907 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.126755 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7kdd\" (UniqueName: \"kubernetes.io/projected/d73705b1-d9d1-45c6-af71-edc45e54e6cb-kube-api-access-n7kdd\") pod \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " Jun 10 17:37:30.126907 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.126791 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-oauth-serving-cert\") pod \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " Jun 10 17:37:30.126907 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.126812 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-service-ca\") pod \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " Jun 10 17:37:30.126907 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.126837 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-oauth-config\") pod \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " Jun 10 17:37:30.127236 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.126916 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-serving-cert\") pod \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\" (UID: \"d73705b1-d9d1-45c6-af71-edc45e54e6cb\") " Jun 10 17:37:30.127236 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.127038 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d73705b1-d9d1-45c6-af71-edc45e54e6cb" (UID: "d73705b1-d9d1-45c6-af71-edc45e54e6cb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:37:30.127236 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.127077 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-config" (OuterVolumeSpecName: "console-config") pod "d73705b1-d9d1-45c6-af71-edc45e54e6cb" (UID: "d73705b1-d9d1-45c6-af71-edc45e54e6cb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:37:30.127236 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.127172 2531 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-trusted-ca-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:37:30.127236 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.127190 2531 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:37:30.127236 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.127206 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d73705b1-d9d1-45c6-af71-edc45e54e6cb" (UID: "d73705b1-d9d1-45c6-af71-edc45e54e6cb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:37:30.127460 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.127251 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-service-ca" (OuterVolumeSpecName: "service-ca") pod "d73705b1-d9d1-45c6-af71-edc45e54e6cb" (UID: "d73705b1-d9d1-45c6-af71-edc45e54e6cb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:37:30.129159 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.129135 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d73705b1-d9d1-45c6-af71-edc45e54e6cb-kube-api-access-n7kdd" (OuterVolumeSpecName: "kube-api-access-n7kdd") pod "d73705b1-d9d1-45c6-af71-edc45e54e6cb" (UID: "d73705b1-d9d1-45c6-af71-edc45e54e6cb"). InnerVolumeSpecName "kube-api-access-n7kdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:37:30.129239 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.129134 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d73705b1-d9d1-45c6-af71-edc45e54e6cb" (UID: "d73705b1-d9d1-45c6-af71-edc45e54e6cb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:37:30.129239 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.129205 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d73705b1-d9d1-45c6-af71-edc45e54e6cb" (UID: "d73705b1-d9d1-45c6-af71-edc45e54e6cb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:37:30.227569 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.227536 2531 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-oauth-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:37:30.227569 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.227562 2531 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d73705b1-d9d1-45c6-af71-edc45e54e6cb-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:37:30.227569 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.227573 2531 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-oauth-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:37:30.227849 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.227583 2531 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d73705b1-d9d1-45c6-af71-edc45e54e6cb-console-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:37:30.227849 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.227591 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7kdd\" (UniqueName: \"kubernetes.io/projected/d73705b1-d9d1-45c6-af71-edc45e54e6cb-kube-api-access-n7kdd\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:37:30.818753 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.818720 2531 generic.go:356] "Generic (PLEG): container finished" podID="d73705b1-d9d1-45c6-af71-edc45e54e6cb" containerID="e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d" exitCode=0 Jun 10 17:37:30.819211 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.818774 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7b777ddd-hcggj" Jun 10 17:37:30.819211 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.818778 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7b777ddd-hcggj" event={"ID":"d73705b1-d9d1-45c6-af71-edc45e54e6cb","Type":"ContainerDied","Data":"e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d"} Jun 10 17:37:30.819211 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.818926 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7b777ddd-hcggj" event={"ID":"d73705b1-d9d1-45c6-af71-edc45e54e6cb","Type":"ContainerDied","Data":"16797055cdeb4fbf24eca0870164f86b1e764d3a6bd32328e7b221285d1711af"} Jun 10 17:37:30.819211 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.818978 2531 scope.go:117] "RemoveContainer" containerID="e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d" Jun 10 17:37:30.826784 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.826770 2531 scope.go:117] "RemoveContainer" containerID="e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d" Jun 10 17:37:30.827074 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:37:30.827054 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d\": container with ID starting with e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d not found: ID does not exist" containerID="e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d" Jun 10 17:37:30.827147 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.827082 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d"} err="failed to get container status \"e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d\": rpc error: code = NotFound desc = could not find container \"e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d\": container with ID starting with e9155dd20cd8758b188327089dd33b9ca04d70e53682f36c21c9665f9fa2370d not found: ID does not exist" Jun 10 17:37:30.832062 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.832042 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7b777ddd-hcggj"] Jun 10 17:37:30.834152 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:30.834133 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c7b777ddd-hcggj"] Jun 10 17:37:32.468506 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:37:32.468467 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d73705b1-d9d1-45c6-af71-edc45e54e6cb" path="/var/lib/kubelet/pods/d73705b1-d9d1-45c6-af71-edc45e54e6cb/volumes" Jun 10 17:38:22.589153 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.589123 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7"] Jun 10 17:38:22.589618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.589365 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73705b1-d9d1-45c6-af71-edc45e54e6cb" containerName="console" Jun 10 17:38:22.589618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.589376 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73705b1-d9d1-45c6-af71-edc45e54e6cb" containerName="console" Jun 10 17:38:22.589618 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.589417 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73705b1-d9d1-45c6-af71-edc45e54e6cb" containerName="console" Jun 10 17:38:22.591979 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.591964 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" Jun 10 17:38:22.594171 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.594148 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 10 17:38:22.594758 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.594739 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 10 17:38:22.594852 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.594768 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-l5xs4\"" Jun 10 17:38:22.602035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.602017 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7"] Jun 10 17:38:22.694452 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.694417 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rl8v7\" (UID: \"4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" Jun 10 17:38:22.694627 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.694465 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw8jg\" (UniqueName: \"kubernetes.io/projected/4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c-kube-api-access-qw8jg\") pod \"openshift-lws-operator-fd8ccff4c-rl8v7\" (UID: \"4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" Jun 10 17:38:22.795616 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.795589 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rl8v7\" (UID: \"4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" Jun 10 17:38:22.795741 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.795631 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw8jg\" (UniqueName: \"kubernetes.io/projected/4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c-kube-api-access-qw8jg\") pod \"openshift-lws-operator-fd8ccff4c-rl8v7\" (UID: \"4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" Jun 10 17:38:22.796020 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.795999 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rl8v7\" (UID: \"4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" Jun 10 17:38:22.804659 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.804640 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw8jg\" (UniqueName: \"kubernetes.io/projected/4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c-kube-api-access-qw8jg\") pod \"openshift-lws-operator-fd8ccff4c-rl8v7\" (UID: \"4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" Jun 10 17:38:22.900671 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:22.900602 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" Jun 10 17:38:23.022363 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:23.022238 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7"] Jun 10 17:38:23.025238 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:38:23.025218 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b7b3d77_f1bf_4f1f_b626_2d54d51cdf4c.slice/crio-4425643a5c86ec23cfe9742d6edc29d3882ee7e319e876a834d74b703848d947 WatchSource:0}: Error finding container 4425643a5c86ec23cfe9742d6edc29d3882ee7e319e876a834d74b703848d947: Status 404 returned error can't find the container with id 4425643a5c86ec23cfe9742d6edc29d3882ee7e319e876a834d74b703848d947 Jun 10 17:38:23.026536 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:23.026516 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 17:38:24.007089 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:24.007002 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" event={"ID":"4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c","Type":"ContainerStarted","Data":"4425643a5c86ec23cfe9742d6edc29d3882ee7e319e876a834d74b703848d947"} Jun 10 17:38:26.016730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:26.016699 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" event={"ID":"4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c","Type":"ContainerStarted","Data":"f219ee6f9e1b4d404603477c47c21eb2651eb813b9ee9a778f39d39ed091188c"} Jun 10 17:38:26.031927 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:26.031882 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl8v7" podStartSLOduration=1.6059114540000001 podStartE2EDuration="4.031865117s" totalImagesPullingTime="2.425953663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:38:26.030716962 +0000 UTC m=+415.998923002" watchObservedRunningTime="2026-06-10 17:38:26.031865117 +0000 UTC m=+416.000071157" Jun 10 17:38:42.103230 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.103197 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh"] Jun 10 17:38:42.107730 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.107711 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.109975 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.109947 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 10 17:38:42.110108 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.110087 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 10 17:38:42.110188 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.110145 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w7vh7\"" Jun 10 17:38:42.110248 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.110195 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 10 17:38:42.110248 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.110227 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 10 17:38:42.125591 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.125570 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh"] Jun 10 17:38:42.242146 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.242121 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-tmp\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.242277 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.242167 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-manifests\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.242277 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.242194 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.242277 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.242250 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.242394 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.242306 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gv8b\" (UniqueName: \"kubernetes.io/projected/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-kube-api-access-2gv8b\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.343320 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.343297 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.343422 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.343328 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gv8b\" (UniqueName: \"kubernetes.io/projected/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-kube-api-access-2gv8b\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.343422 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.343356 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-tmp\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.343422 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.343394 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-manifests\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.343569 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.343421 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.343883 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.343857 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-manifests\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.345744 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.345722 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-tmp\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.345951 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.345916 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.346042 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.346027 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.356980 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.356919 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gv8b\" (UniqueName: \"kubernetes.io/projected/cdb0f390-27ea-4e36-a5d4-2c58145a44e0-kube-api-access-2gv8b\") pod \"opendatahub-operator-controller-manager-55bf9956cc-gv2bh\" (UID: \"cdb0f390-27ea-4e36-a5d4-2c58145a44e0\") " pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.417760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.417739 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:42.549357 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:42.549254 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh"] Jun 10 17:38:42.552150 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:38:42.552124 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb0f390_27ea_4e36_a5d4_2c58145a44e0.slice/crio-8b1383ef875b11d4ad5fe2f34e0aeb26c60d3745d39c069ab0dd82d8286b5a69 WatchSource:0}: Error finding container 8b1383ef875b11d4ad5fe2f34e0aeb26c60d3745d39c069ab0dd82d8286b5a69: Status 404 returned error can't find the container with id 8b1383ef875b11d4ad5fe2f34e0aeb26c60d3745d39c069ab0dd82d8286b5a69 Jun 10 17:38:43.075469 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:43.075431 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" event={"ID":"cdb0f390-27ea-4e36-a5d4-2c58145a44e0","Type":"ContainerStarted","Data":"8b1383ef875b11d4ad5fe2f34e0aeb26c60d3745d39c069ab0dd82d8286b5a69"} Jun 10 17:38:46.089179 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:46.089142 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" event={"ID":"cdb0f390-27ea-4e36-a5d4-2c58145a44e0","Type":"ContainerStarted","Data":"0e08365433efaae6f2384425442a32aa617839acfabd15586681f28a2601c1bb"} Jun 10 17:38:47.094604 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:47.094571 2531 generic.go:356] "Generic (PLEG): container finished" podID="cdb0f390-27ea-4e36-a5d4-2c58145a44e0" containerID="0e08365433efaae6f2384425442a32aa617839acfabd15586681f28a2601c1bb" exitCode=0 Jun 10 17:38:47.094604 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:47.094607 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" event={"ID":"cdb0f390-27ea-4e36-a5d4-2c58145a44e0","Type":"ContainerDied","Data":"0e08365433efaae6f2384425442a32aa617839acfabd15586681f28a2601c1bb"} Jun 10 17:38:48.100324 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:48.100250 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" event={"ID":"cdb0f390-27ea-4e36-a5d4-2c58145a44e0","Type":"ContainerStarted","Data":"9ce2f670ddfac7fe8ccf9202124b440c36c9c02d6573e7e47be5825b08011646"} Jun 10 17:38:48.100643 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:48.100475 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:38:48.125113 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:48.125056 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" podStartSLOduration=2.119803685 podStartE2EDuration="6.125038631s" totalImagesPullingTime="3.025909166s" totalInitContainerRuntime="979.32578ms" isStatefulPod=false podCreationTimestamp="2026-06-10 17:38:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:38:48.12351486 +0000 UTC m=+438.091720923" watchObservedRunningTime="2026-06-10 17:38:48.125038631 +0000 UTC m=+438.093244673" Jun 10 17:38:59.106798 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:38:59.106766 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55bf9956cc-gv2bh" Jun 10 17:39:01.144785 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.144744 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx"] Jun 10 17:39:01.243047 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.243018 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx"] Jun 10 17:39:01.243192 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.243113 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.246498 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.246477 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 10 17:39:01.246620 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.246524 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 10 17:39:01.246620 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.246525 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tjbpt\"" Jun 10 17:39:01.246620 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.246557 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 10 17:39:01.385371 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.385345 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/caf6a2d1-1415-4a88-8333-3555a330c937-manager-config\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.385525 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.385389 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmqzk\" (UniqueName: \"kubernetes.io/projected/caf6a2d1-1415-4a88-8333-3555a330c937-kube-api-access-rmqzk\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.385525 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.385468 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/caf6a2d1-1415-4a88-8333-3555a330c937-metrics-cert\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.385525 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.385489 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/caf6a2d1-1415-4a88-8333-3555a330c937-cert\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.486321 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.486248 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/caf6a2d1-1415-4a88-8333-3555a330c937-metrics-cert\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.486321 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.486279 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/caf6a2d1-1415-4a88-8333-3555a330c937-cert\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.486321 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.486311 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/caf6a2d1-1415-4a88-8333-3555a330c937-manager-config\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.486583 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.486358 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmqzk\" (UniqueName: \"kubernetes.io/projected/caf6a2d1-1415-4a88-8333-3555a330c937-kube-api-access-rmqzk\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.487011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.486988 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/caf6a2d1-1415-4a88-8333-3555a330c937-manager-config\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.488726 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.488702 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/caf6a2d1-1415-4a88-8333-3555a330c937-metrics-cert\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.488823 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.488746 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/caf6a2d1-1415-4a88-8333-3555a330c937-cert\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.495645 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.495623 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmqzk\" (UniqueName: \"kubernetes.io/projected/caf6a2d1-1415-4a88-8333-3555a330c937-kube-api-access-rmqzk\") pod \"lws-controller-manager-6648fffc94-nttzx\" (UID: \"caf6a2d1-1415-4a88-8333-3555a330c937\") " pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.552521 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.552496 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:01.673979 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:01.673899 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx"] Jun 10 17:39:01.676334 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:39:01.676313 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf6a2d1_1415_4a88_8333_3555a330c937.slice/crio-522c480a99f66cd451fd54e42fa60a00c0e36f6c935c465164d1e45a6a906ff4 WatchSource:0}: Error finding container 522c480a99f66cd451fd54e42fa60a00c0e36f6c935c465164d1e45a6a906ff4: Status 404 returned error can't find the container with id 522c480a99f66cd451fd54e42fa60a00c0e36f6c935c465164d1e45a6a906ff4 Jun 10 17:39:02.152335 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:02.152298 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" event={"ID":"caf6a2d1-1415-4a88-8333-3555a330c937","Type":"ContainerStarted","Data":"522c480a99f66cd451fd54e42fa60a00c0e36f6c935c465164d1e45a6a906ff4"} Jun 10 17:39:05.165901 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:05.165863 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" event={"ID":"caf6a2d1-1415-4a88-8333-3555a330c937","Type":"ContainerStarted","Data":"9f4bb9316f7515f48e01fc524375415d51edd6e4ce5adae0347b2a82ca85274f"} Jun 10 17:39:05.166375 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:05.165991 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:05.182215 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:05.182170 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" podStartSLOduration=1.607603647 podStartE2EDuration="4.182157561s" totalImagesPullingTime="2.574553914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:39:05.180354776 +0000 UTC m=+455.148560815" watchObservedRunningTime="2026-06-10 17:39:05.182157561 +0000 UTC m=+455.150363599" Jun 10 17:39:16.171567 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:16.171537 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6648fffc94-nttzx" Jun 10 17:39:20.435559 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.435523 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-7474f67696-477n8"] Jun 10 17:39:20.439092 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.439073 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7474f67696-477n8" Jun 10 17:39:20.441007 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.440981 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7474f67696-477n8"] Jun 10 17:39:20.441307 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.441290 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 10 17:39:20.441409 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.441393 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-52qpd\"" Jun 10 17:39:20.527138 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.527100 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l8gb\" (UniqueName: \"kubernetes.io/projected/577450d8-2a8e-4754-ba87-02c2053599e7-kube-api-access-7l8gb\") pod \"dashboard-redirect-7474f67696-477n8\" (UID: \"577450d8-2a8e-4754-ba87-02c2053599e7\") " pod="opendatahub/dashboard-redirect-7474f67696-477n8" Jun 10 17:39:20.527301 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.527145 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/577450d8-2a8e-4754-ba87-02c2053599e7-redirect-config\") pod \"dashboard-redirect-7474f67696-477n8\" (UID: \"577450d8-2a8e-4754-ba87-02c2053599e7\") " pod="opendatahub/dashboard-redirect-7474f67696-477n8" Jun 10 17:39:20.627904 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.627857 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/577450d8-2a8e-4754-ba87-02c2053599e7-redirect-config\") pod \"dashboard-redirect-7474f67696-477n8\" (UID: \"577450d8-2a8e-4754-ba87-02c2053599e7\") " pod="opendatahub/dashboard-redirect-7474f67696-477n8" Jun 10 17:39:20.628079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.627964 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l8gb\" (UniqueName: \"kubernetes.io/projected/577450d8-2a8e-4754-ba87-02c2053599e7-kube-api-access-7l8gb\") pod \"dashboard-redirect-7474f67696-477n8\" (UID: \"577450d8-2a8e-4754-ba87-02c2053599e7\") " pod="opendatahub/dashboard-redirect-7474f67696-477n8" Jun 10 17:39:20.628565 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.628547 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/577450d8-2a8e-4754-ba87-02c2053599e7-redirect-config\") pod \"dashboard-redirect-7474f67696-477n8\" (UID: \"577450d8-2a8e-4754-ba87-02c2053599e7\") " pod="opendatahub/dashboard-redirect-7474f67696-477n8" Jun 10 17:39:20.638291 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.638256 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l8gb\" (UniqueName: \"kubernetes.io/projected/577450d8-2a8e-4754-ba87-02c2053599e7-kube-api-access-7l8gb\") pod \"dashboard-redirect-7474f67696-477n8\" (UID: \"577450d8-2a8e-4754-ba87-02c2053599e7\") " pod="opendatahub/dashboard-redirect-7474f67696-477n8" Jun 10 17:39:20.750167 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.750148 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7474f67696-477n8" Jun 10 17:39:20.874190 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:20.874161 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7474f67696-477n8"] Jun 10 17:39:20.877110 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:39:20.877078 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577450d8_2a8e_4754_ba87_02c2053599e7.slice/crio-0f190b6a39242ffe87e17d79350d4e260694aa6c230ade695325afd4aa376c29 WatchSource:0}: Error finding container 0f190b6a39242ffe87e17d79350d4e260694aa6c230ade695325afd4aa376c29: Status 404 returned error can't find the container with id 0f190b6a39242ffe87e17d79350d4e260694aa6c230ade695325afd4aa376c29 Jun 10 17:39:21.230879 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:21.230790 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7474f67696-477n8" event={"ID":"577450d8-2a8e-4754-ba87-02c2053599e7","Type":"ContainerStarted","Data":"0f190b6a39242ffe87e17d79350d4e260694aa6c230ade695325afd4aa376c29"} Jun 10 17:39:26.256530 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:26.256493 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7474f67696-477n8" event={"ID":"577450d8-2a8e-4754-ba87-02c2053599e7","Type":"ContainerStarted","Data":"e93f490eda8688c7206cdbe750a2756b2f524d27c70b3ea0dbe6e687b56e1c70"} Jun 10 17:39:26.276146 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:26.276062 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-7474f67696-477n8" podStartSLOduration=1.141504896 podStartE2EDuration="6.276048069s" totalImagesPullingTime="5.134543173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:39:26.275809033 +0000 UTC m=+476.244015071" watchObservedRunningTime="2026-06-10 17:39:26.276048069 +0000 UTC m=+476.244254125" Jun 10 17:39:38.539849 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.539805 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq"] Jun 10 17:39:38.543363 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.543345 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.545583 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.545551 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fk4cr\"" Jun 10 17:39:38.545693 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.545607 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 10 17:39:38.545693 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.545628 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 10 17:39:38.545693 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.545645 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 10 17:39:38.554097 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.554074 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq"] Jun 10 17:39:38.675011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.674975 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.675174 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.675031 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.675174 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.675081 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.675174 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.675114 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.675174 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.675155 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.675345 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.675195 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.675345 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.675237 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z857k\" (UniqueName: \"kubernetes.io/projected/c9ffd22f-8d03-45ec-a2e0-867870c17f66-kube-api-access-z857k\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.675345 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.675291 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.675345 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.675309 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776144 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776113 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776302 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776151 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776302 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776180 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776302 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776205 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776302 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776242 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776302 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776268 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z857k\" (UniqueName: \"kubernetes.io/projected/c9ffd22f-8d03-45ec-a2e0-867870c17f66-kube-api-access-z857k\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776576 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776304 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776576 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776325 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776576 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776365 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.776905 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.776631 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.777040 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.777021 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.777108 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.777080 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.777108 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.777091 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.777200 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.777139 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.778894 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.778875 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.779144 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.779127 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.784236 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.784215 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ffd22f-8d03-45ec-a2e0-867870c17f66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.784801 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.784768 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z857k\" (UniqueName: \"kubernetes.io/projected/c9ffd22f-8d03-45ec-a2e0-867870c17f66-kube-api-access-z857k\") pod \"data-science-gateway-data-science-gateway-class-74759f9795cfcqq\" (UID: \"c9ffd22f-8d03-45ec-a2e0-867870c17f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:38.852441 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:38.852369 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:39.012440 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:39.012341 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq"] Jun 10 17:39:39.015479 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:39:39.015453 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ffd22f_8d03_45ec_a2e0_867870c17f66.slice/crio-598ed450fe447b09aa20035573442723a7062b9f43d814b2b8293b37d0065557 WatchSource:0}: Error finding container 598ed450fe447b09aa20035573442723a7062b9f43d814b2b8293b37d0065557: Status 404 returned error can't find the container with id 598ed450fe447b09aa20035573442723a7062b9f43d814b2b8293b37d0065557 Jun 10 17:39:39.308485 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:39.308449 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" event={"ID":"c9ffd22f-8d03-45ec-a2e0-867870c17f66","Type":"ContainerStarted","Data":"598ed450fe447b09aa20035573442723a7062b9f43d814b2b8293b37d0065557"} Jun 10 17:39:41.417453 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:41.417410 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:39:41.417723 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:41.417487 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:39:41.417723 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:41.417519 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:39:42.323180 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:42.323144 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" event={"ID":"c9ffd22f-8d03-45ec-a2e0-867870c17f66","Type":"ContainerStarted","Data":"64ec2a0c15795029c420490abafc83c23d5ed7e4c7bfcdbf1ed59462f402e1dc"} Jun 10 17:39:42.345476 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:42.345412 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" podStartSLOduration=1.9457108509999999 podStartE2EDuration="4.345396447s" totalImagesPullingTime="2.399685596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:39:42.343352975 +0000 UTC m=+492.311559011" watchObservedRunningTime="2026-06-10 17:39:42.345396447 +0000 UTC m=+492.313602489" Jun 10 17:39:42.853523 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:42.853491 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:42.858038 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:42.858015 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:43.327889 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:43.327861 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:39:43.328898 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:39:43.328877 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795cfcqq" Jun 10 17:40:02.436123 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.436089 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rjjtb"] Jun 10 17:40:02.456233 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.456199 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rjjtb"] Jun 10 17:40:02.456373 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.456285 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" Jun 10 17:40:02.458775 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.458740 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 10 17:40:02.458909 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.458790 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 10 17:40:02.459526 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.459490 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-pmz4j\"" Jun 10 17:40:02.572607 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.572577 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tnfc\" (UniqueName: \"kubernetes.io/projected/21997db7-6098-4526-aa22-b8fcc739ac2f-kube-api-access-2tnfc\") pod \"kuadrant-operator-catalog-rjjtb\" (UID: \"21997db7-6098-4526-aa22-b8fcc739ac2f\") " pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" Jun 10 17:40:02.673339 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.673309 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tnfc\" (UniqueName: \"kubernetes.io/projected/21997db7-6098-4526-aa22-b8fcc739ac2f-kube-api-access-2tnfc\") pod \"kuadrant-operator-catalog-rjjtb\" (UID: \"21997db7-6098-4526-aa22-b8fcc739ac2f\") " pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" Jun 10 17:40:02.681133 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.681105 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tnfc\" (UniqueName: \"kubernetes.io/projected/21997db7-6098-4526-aa22-b8fcc739ac2f-kube-api-access-2tnfc\") pod \"kuadrant-operator-catalog-rjjtb\" (UID: \"21997db7-6098-4526-aa22-b8fcc739ac2f\") " pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" Jun 10 17:40:02.767172 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.767147 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" Jun 10 17:40:02.890001 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.889979 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rjjtb"] Jun 10 17:40:02.891866 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:40:02.891836 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21997db7_6098_4526_aa22_b8fcc739ac2f.slice/crio-0a6f49a5830fd1b37582146af9f284e582d1a79fdf6de462a5b17e533a206717 WatchSource:0}: Error finding container 0a6f49a5830fd1b37582146af9f284e582d1a79fdf6de462a5b17e533a206717: Status 404 returned error can't find the container with id 0a6f49a5830fd1b37582146af9f284e582d1a79fdf6de462a5b17e533a206717 Jun 10 17:40:02.935573 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:02.935550 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rjjtb"] Jun 10 17:40:03.013086 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.013064 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2pvh"] Jun 10 17:40:03.018613 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.018311 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:03.023826 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.023806 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2pvh"] Jun 10 17:40:03.176583 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.176547 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdm8q\" (UniqueName: \"kubernetes.io/projected/7042b236-9910-4ce7-9dab-2278053d89a5-kube-api-access-sdm8q\") pod \"kuadrant-operator-catalog-b2pvh\" (UID: \"7042b236-9910-4ce7-9dab-2278053d89a5\") " pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:03.277930 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.277851 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdm8q\" (UniqueName: \"kubernetes.io/projected/7042b236-9910-4ce7-9dab-2278053d89a5-kube-api-access-sdm8q\") pod \"kuadrant-operator-catalog-b2pvh\" (UID: \"7042b236-9910-4ce7-9dab-2278053d89a5\") " pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:03.285228 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.285206 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdm8q\" (UniqueName: \"kubernetes.io/projected/7042b236-9910-4ce7-9dab-2278053d89a5-kube-api-access-sdm8q\") pod \"kuadrant-operator-catalog-b2pvh\" (UID: \"7042b236-9910-4ce7-9dab-2278053d89a5\") " pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:03.329141 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.329114 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:03.409215 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.409009 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" event={"ID":"21997db7-6098-4526-aa22-b8fcc739ac2f","Type":"ContainerStarted","Data":"0a6f49a5830fd1b37582146af9f284e582d1a79fdf6de462a5b17e533a206717"} Jun 10 17:40:03.454449 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:03.454424 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2pvh"] Jun 10 17:40:03.456323 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:40:03.456297 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7042b236_9910_4ce7_9dab_2278053d89a5.slice/crio-477e898cbaf9d4a3c7c1f6170452e6cedc80b94a2c3b4f3e68e84db8e13e7417 WatchSource:0}: Error finding container 477e898cbaf9d4a3c7c1f6170452e6cedc80b94a2c3b4f3e68e84db8e13e7417: Status 404 returned error can't find the container with id 477e898cbaf9d4a3c7c1f6170452e6cedc80b94a2c3b4f3e68e84db8e13e7417 Jun 10 17:40:04.415648 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:04.415613 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" event={"ID":"7042b236-9910-4ce7-9dab-2278053d89a5","Type":"ContainerStarted","Data":"477e898cbaf9d4a3c7c1f6170452e6cedc80b94a2c3b4f3e68e84db8e13e7417"} Jun 10 17:40:05.421742 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.421647 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" event={"ID":"7042b236-9910-4ce7-9dab-2278053d89a5","Type":"ContainerStarted","Data":"c9d983f8b37a8d02291494f9bd228d26546f9ce0d534e486f88309f7e163fdfa"} Jun 10 17:40:05.423076 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.423054 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" event={"ID":"21997db7-6098-4526-aa22-b8fcc739ac2f","Type":"ContainerStarted","Data":"0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47"} Jun 10 17:40:05.423191 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.423117 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" podUID="21997db7-6098-4526-aa22-b8fcc739ac2f" containerName="registry-server" containerID="cri-o://0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47" gracePeriod=2 Jun 10 17:40:05.436578 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.436535 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" podStartSLOduration=1.687280995 podStartE2EDuration="3.436520025s" totalImagesPullingTime="1.74923903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:40:05.434764756 +0000 UTC m=+515.402970814" watchObservedRunningTime="2026-06-10 17:40:05.436520025 +0000 UTC m=+515.404726065" Jun 10 17:40:05.448896 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.448857 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" podStartSLOduration=1.199470509 podStartE2EDuration="3.448847447s" totalImagesPullingTime="2.249376938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:40:05.44761011 +0000 UTC m=+515.415816151" watchObservedRunningTime="2026-06-10 17:40:05.448847447 +0000 UTC m=+515.417053485" Jun 10 17:40:05.660295 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.660273 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" Jun 10 17:40:05.696880 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.696817 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tnfc\" (UniqueName: \"kubernetes.io/projected/21997db7-6098-4526-aa22-b8fcc739ac2f-kube-api-access-2tnfc\") pod \"21997db7-6098-4526-aa22-b8fcc739ac2f\" (UID: \"21997db7-6098-4526-aa22-b8fcc739ac2f\") " Jun 10 17:40:05.699000 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.698974 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21997db7-6098-4526-aa22-b8fcc739ac2f-kube-api-access-2tnfc" (OuterVolumeSpecName: "kube-api-access-2tnfc") pod "21997db7-6098-4526-aa22-b8fcc739ac2f" (UID: "21997db7-6098-4526-aa22-b8fcc739ac2f"). InnerVolumeSpecName "kube-api-access-2tnfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:40:05.797700 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:05.797670 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tnfc\" (UniqueName: \"kubernetes.io/projected/21997db7-6098-4526-aa22-b8fcc739ac2f-kube-api-access-2tnfc\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:40:06.428624 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.428532 2531 generic.go:356] "Generic (PLEG): container finished" podID="21997db7-6098-4526-aa22-b8fcc739ac2f" containerID="0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47" exitCode=0 Jun 10 17:40:06.428624 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.428577 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" Jun 10 17:40:06.429106 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.428619 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" event={"ID":"21997db7-6098-4526-aa22-b8fcc739ac2f","Type":"ContainerDied","Data":"0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47"} Jun 10 17:40:06.429106 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.428670 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rjjtb" event={"ID":"21997db7-6098-4526-aa22-b8fcc739ac2f","Type":"ContainerDied","Data":"0a6f49a5830fd1b37582146af9f284e582d1a79fdf6de462a5b17e533a206717"} Jun 10 17:40:06.429106 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.428691 2531 scope.go:117] "RemoveContainer" containerID="0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47" Jun 10 17:40:06.437755 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.437731 2531 scope.go:117] "RemoveContainer" containerID="0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47" Jun 10 17:40:06.438024 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:40:06.438006 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47\": container with ID starting with 0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47 not found: ID does not exist" containerID="0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47" Jun 10 17:40:06.438081 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.438032 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47"} err="failed to get container status \"0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47\": rpc error: code = NotFound desc = could not find container \"0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47\": container with ID starting with 0597ce0bc50f966a58a17464f077032b2e8e8a2f09edc1caebc54c38dc69ad47 not found: ID does not exist" Jun 10 17:40:06.447262 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.447240 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rjjtb"] Jun 10 17:40:06.449230 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.449210 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rjjtb"] Jun 10 17:40:06.469322 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:06.469304 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21997db7-6098-4526-aa22-b8fcc739ac2f" path="/var/lib/kubelet/pods/21997db7-6098-4526-aa22-b8fcc739ac2f/volumes" Jun 10 17:40:13.329840 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:13.329798 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:13.330255 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:13.329853 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:13.352051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:13.352025 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:13.477989 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:13.477967 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-b2pvh" Jun 10 17:40:34.864119 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:34.864081 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd"] Jun 10 17:40:34.864556 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:34.864358 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21997db7-6098-4526-aa22-b8fcc739ac2f" containerName="registry-server" Jun 10 17:40:34.864556 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:34.864369 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="21997db7-6098-4526-aa22-b8fcc739ac2f" containerName="registry-server" Jun 10 17:40:34.864556 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:34.864431 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="21997db7-6098-4526-aa22-b8fcc739ac2f" containerName="registry-server" Jun 10 17:40:34.868519 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:34.868501 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:40:34.870860 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:34.870840 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-js8rx\"" Jun 10 17:40:34.876189 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:34.876165 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd"] Jun 10 17:40:34.918696 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:34.918668 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4hnf\" (UniqueName: \"kubernetes.io/projected/7e9897b3-7f71-4a36-a97c-9e32a39d4d03-kube-api-access-p4hnf\") pod \"limitador-operator-controller-manager-859d7b78bd-wb5jd\" (UID: \"7e9897b3-7f71-4a36-a97c-9e32a39d4d03\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:40:35.019200 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:35.019168 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4hnf\" (UniqueName: \"kubernetes.io/projected/7e9897b3-7f71-4a36-a97c-9e32a39d4d03-kube-api-access-p4hnf\") pod \"limitador-operator-controller-manager-859d7b78bd-wb5jd\" (UID: \"7e9897b3-7f71-4a36-a97c-9e32a39d4d03\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:40:35.034049 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:35.034023 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4hnf\" (UniqueName: \"kubernetes.io/projected/7e9897b3-7f71-4a36-a97c-9e32a39d4d03-kube-api-access-p4hnf\") pod \"limitador-operator-controller-manager-859d7b78bd-wb5jd\" (UID: \"7e9897b3-7f71-4a36-a97c-9e32a39d4d03\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:40:35.178603 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:35.178527 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:40:35.310570 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:35.310539 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd"] Jun 10 17:40:35.314541 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:40:35.314516 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e9897b3_7f71_4a36_a97c_9e32a39d4d03.slice/crio-f7a23be56535ab07ac93085b24fe4b77cb20552e55c6bf0449f4af67080bc346 WatchSource:0}: Error finding container f7a23be56535ab07ac93085b24fe4b77cb20552e55c6bf0449f4af67080bc346: Status 404 returned error can't find the container with id f7a23be56535ab07ac93085b24fe4b77cb20552e55c6bf0449f4af67080bc346 Jun 10 17:40:35.539637 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:35.539607 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" event={"ID":"7e9897b3-7f71-4a36-a97c-9e32a39d4d03","Type":"ContainerStarted","Data":"f7a23be56535ab07ac93085b24fe4b77cb20552e55c6bf0449f4af67080bc346"} Jun 10 17:40:38.553830 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:38.553796 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" event={"ID":"7e9897b3-7f71-4a36-a97c-9e32a39d4d03","Type":"ContainerStarted","Data":"812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0"} Jun 10 17:40:38.554218 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:38.553923 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:40:38.577035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:38.576986 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" podStartSLOduration=2.0616571 podStartE2EDuration="4.576972332s" totalImagesPullingTime="2.515315232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:40:38.576852254 +0000 UTC m=+548.545058304" watchObservedRunningTime="2026-06-10 17:40:38.576972332 +0000 UTC m=+548.545178371" Jun 10 17:40:40.842893 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:40.842858 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2"] Jun 10 17:40:40.846326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:40.846304 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:40.849120 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:40.849102 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-npmch\"" Jun 10 17:40:40.859904 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:40.859884 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2"] Jun 10 17:40:40.975298 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:40.975264 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" (UID: \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:40.975431 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:40.975314 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cvnz\" (UniqueName: \"kubernetes.io/projected/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-kube-api-access-9cvnz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" (UID: \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:41.076061 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:41.076021 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" (UID: \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:41.076160 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:41.076078 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cvnz\" (UniqueName: \"kubernetes.io/projected/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-kube-api-access-9cvnz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" (UID: \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:41.076384 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:41.076364 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" (UID: \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:41.089646 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:41.089620 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cvnz\" (UniqueName: \"kubernetes.io/projected/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-kube-api-access-9cvnz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" (UID: \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:41.155469 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:41.155415 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:41.302063 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:41.302032 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2"] Jun 10 17:40:41.305091 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:40:41.305065 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2d4c07_8c7f_4f59_a9a1_a87500f007f0.slice/crio-10942bb773ca1ac581c211e9a44ec510dc4e48461407eaf419b871a10be27dd1 WatchSource:0}: Error finding container 10942bb773ca1ac581c211e9a44ec510dc4e48461407eaf419b871a10be27dd1: Status 404 returned error can't find the container with id 10942bb773ca1ac581c211e9a44ec510dc4e48461407eaf419b871a10be27dd1 Jun 10 17:40:41.567247 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:41.567217 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" event={"ID":"db2d4c07-8c7f-4f59-a9a1-a87500f007f0","Type":"ContainerStarted","Data":"10942bb773ca1ac581c211e9a44ec510dc4e48461407eaf419b871a10be27dd1"} Jun 10 17:40:48.293742 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.293708 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65447df49f-nnlzl"] Jun 10 17:40:48.296966 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.296949 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.309716 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.309692 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65447df49f-nnlzl"] Jun 10 17:40:48.440235 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.440197 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-serving-cert\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.440235 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.440231 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvdh7\" (UniqueName: \"kubernetes.io/projected/d074da7d-d94c-4bdd-86f6-645c2aea4770-kube-api-access-zvdh7\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.440417 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.440252 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-oauth-config\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.440417 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.440311 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-oauth-serving-cert\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.440417 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.440350 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-trusted-ca-bundle\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.440417 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.440395 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-service-ca\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.440538 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.440459 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-config\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.541362 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.541329 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-config\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.541362 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.541365 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-serving-cert\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.541587 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.541380 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvdh7\" (UniqueName: \"kubernetes.io/projected/d074da7d-d94c-4bdd-86f6-645c2aea4770-kube-api-access-zvdh7\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.541587 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.541398 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-oauth-config\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.541587 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.541419 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-oauth-serving-cert\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.541722 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.541629 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-trusted-ca-bundle\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.541722 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.541677 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-service-ca\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.542240 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.542212 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-config\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.542364 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.542319 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-service-ca\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.542435 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.542418 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-trusted-ca-bundle\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.542477 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.542426 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d074da7d-d94c-4bdd-86f6-645c2aea4770-oauth-serving-cert\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.544032 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.543979 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-serving-cert\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.544122 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.544087 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d074da7d-d94c-4bdd-86f6-645c2aea4770-console-oauth-config\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.548819 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.548797 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvdh7\" (UniqueName: \"kubernetes.io/projected/d074da7d-d94c-4bdd-86f6-645c2aea4770-kube-api-access-zvdh7\") pod \"console-65447df49f-nnlzl\" (UID: \"d074da7d-d94c-4bdd-86f6-645c2aea4770\") " pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.598253 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.598225 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" event={"ID":"db2d4c07-8c7f-4f59-a9a1-a87500f007f0","Type":"ContainerStarted","Data":"d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac"} Jun 10 17:40:48.598381 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.598365 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:40:48.606274 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.606252 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:48.735018 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.734891 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" podStartSLOduration=2.164776236 podStartE2EDuration="8.734872556s" totalImagesPullingTime="6.57009632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:40:48.62783151 +0000 UTC m=+558.596037564" watchObservedRunningTime="2026-06-10 17:40:48.734872556 +0000 UTC m=+558.703078595" Jun 10 17:40:48.735115 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:48.735054 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65447df49f-nnlzl"] Jun 10 17:40:48.737867 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:40:48.737839 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd074da7d_d94c_4bdd_86f6_645c2aea4770.slice/crio-7fcb58f7f2a567229f214a53385ba643a983c2c11912fb143eb1fc16be04d877 WatchSource:0}: Error finding container 7fcb58f7f2a567229f214a53385ba643a983c2c11912fb143eb1fc16be04d877: Status 404 returned error can't find the container with id 7fcb58f7f2a567229f214a53385ba643a983c2c11912fb143eb1fc16be04d877 Jun 10 17:40:49.560066 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:49.560031 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:40:49.609856 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:49.609823 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65447df49f-nnlzl" event={"ID":"d074da7d-d94c-4bdd-86f6-645c2aea4770","Type":"ContainerStarted","Data":"b40501a45e4d72416e6f1cd365f813f30ffd2d7face61d1a72e107e14d4fda21"} Jun 10 17:40:49.610059 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:49.609900 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65447df49f-nnlzl" event={"ID":"d074da7d-d94c-4bdd-86f6-645c2aea4770","Type":"ContainerStarted","Data":"7fcb58f7f2a567229f214a53385ba643a983c2c11912fb143eb1fc16be04d877"} Jun 10 17:40:49.626657 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:49.626605 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65447df49f-nnlzl" podStartSLOduration=1.626586107 podStartE2EDuration="1.626586107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:40:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:40:49.625604066 +0000 UTC m=+559.593810117" watchObservedRunningTime="2026-06-10 17:40:49.626586107 +0000 UTC m=+559.594792150" Jun 10 17:40:50.569826 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.569798 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g"] Jun 10 17:40:50.573455 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.573434 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:50.575649 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.575628 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d2rp2\"" Jun 10 17:40:50.575649 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.575644 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 10 17:40:50.575829 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.575666 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 10 17:40:50.583401 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.583375 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g"] Jun 10 17:40:50.661673 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.661645 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:50.661673 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.661675 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbt6r\" (UniqueName: \"kubernetes.io/projected/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-kube-api-access-bbt6r\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:50.661878 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.661714 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:50.762098 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.762066 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:50.762262 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.762109 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbt6r\" (UniqueName: \"kubernetes.io/projected/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-kube-api-access-bbt6r\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:50.762262 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.762153 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:50.762377 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:40:50.762301 2531 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 10 17:40:50.762377 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:40:50.762368 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-plugin-serving-cert podName:15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd nodeName:}" failed. No retries permitted until 2026-06-10 17:40:51.262346967 +0000 UTC m=+561.230553002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-pjl5g" (UID: "15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd") : secret "plugin-serving-cert" not found Jun 10 17:40:50.763135 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.763113 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:50.770509 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:50.770491 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbt6r\" (UniqueName: \"kubernetes.io/projected/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-kube-api-access-bbt6r\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:51.267059 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:51.267014 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:51.269662 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:51.269638 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pjl5g\" (UID: \"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:51.483334 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:51.483297 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" Jun 10 17:40:51.609420 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:51.609398 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g"] Jun 10 17:40:51.611576 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:40:51.611548 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15f81e6c_bbd7_4d60_a5e8_b372ca4b97dd.slice/crio-d6093cc28688585cc8e2591d1e5430a5451baa060639159ac5797ec434bb4fd9 WatchSource:0}: Error finding container d6093cc28688585cc8e2591d1e5430a5451baa060639159ac5797ec434bb4fd9: Status 404 returned error can't find the container with id d6093cc28688585cc8e2591d1e5430a5451baa060639159ac5797ec434bb4fd9 Jun 10 17:40:51.620925 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:51.620902 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" event={"ID":"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd","Type":"ContainerStarted","Data":"d6093cc28688585cc8e2591d1e5430a5451baa060639159ac5797ec434bb4fd9"} Jun 10 17:40:58.606624 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:58.606589 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:58.607106 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:58.606655 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:58.611275 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:58.611257 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:58.659920 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:58.659890 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65447df49f-nnlzl" Jun 10 17:40:58.708089 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:58.708056 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c784d7ffb-smwcx"] Jun 10 17:40:59.612341 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:40:59.612313 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:41:01.297830 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.297791 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2"] Jun 10 17:41:01.298267 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.298096 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" containerName="manager" containerID="cri-o://d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac" gracePeriod=2 Jun 10 17:41:01.309728 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.309690 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2"] Jun 10 17:41:01.310551 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:01.310510 2531 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\": pods \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:41:01.319766 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.319736 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd"] Jun 10 17:41:01.320163 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.319992 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" containerName="manager" containerID="cri-o://812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0" gracePeriod=2 Jun 10 17:41:01.324337 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.324314 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt"] Jun 10 17:41:01.324758 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.324735 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" containerName="manager" Jun 10 17:41:01.324844 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.324760 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" containerName="manager" Jun 10 17:41:01.324891 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.324849 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" containerName="manager" Jun 10 17:41:01.331166 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.331143 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:01.333826 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.333800 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd"] Jun 10 17:41:01.342165 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.342143 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt"] Jun 10 17:41:01.346713 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:01.346682 2531 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7e9897b3-7f71-4a36-a97c-9e32a39d4d03\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-wb5jd\": pods \"limitador-operator-controller-manager-859d7b78bd-wb5jd\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:41:01.348605 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:01.348573 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:41:01.350303 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:01.350272 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wb5jd\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:41:01.351890 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:01.351861 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:41:01.458802 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.458766 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvqsc\" (UniqueName: \"kubernetes.io/projected/185b864b-5a36-44b5-89b1-eb0f58716a38-kube-api-access-wvqsc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5rdwt\" (UID: \"185b864b-5a36-44b5-89b1-eb0f58716a38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:01.458972 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.458863 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/185b864b-5a36-44b5-89b1-eb0f58716a38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5rdwt\" (UID: \"185b864b-5a36-44b5-89b1-eb0f58716a38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:01.559967 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.559868 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/185b864b-5a36-44b5-89b1-eb0f58716a38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5rdwt\" (UID: \"185b864b-5a36-44b5-89b1-eb0f58716a38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:01.560131 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.560016 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvqsc\" (UniqueName: \"kubernetes.io/projected/185b864b-5a36-44b5-89b1-eb0f58716a38-kube-api-access-wvqsc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5rdwt\" (UID: \"185b864b-5a36-44b5-89b1-eb0f58716a38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:01.560341 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.560318 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/185b864b-5a36-44b5-89b1-eb0f58716a38-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5rdwt\" (UID: \"185b864b-5a36-44b5-89b1-eb0f58716a38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:01.584370 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.584343 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvqsc\" (UniqueName: \"kubernetes.io/projected/185b864b-5a36-44b5-89b1-eb0f58716a38-kube-api-access-wvqsc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5rdwt\" (UID: \"185b864b-5a36-44b5-89b1-eb0f58716a38\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:01.722719 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:01.722687 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:13.938262 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.938234 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:41:13.940069 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:13.940044 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wb5jd\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:41:13.941049 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.941035 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:41:13.942507 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:13.942486 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:41:13.944027 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:13.943917 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wb5jd\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:41:13.952121 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.949592 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt"] Jun 10 17:41:13.952121 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:41:13.950389 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185b864b_5a36_44b5_89b1_eb0f58716a38.slice/crio-8368d2aa33431f6e562bc94b81146959021575393e08e032daf469b57bde2fd8 WatchSource:0}: Error finding container 8368d2aa33431f6e562bc94b81146959021575393e08e032daf469b57bde2fd8: Status 404 returned error can't find the container with id 8368d2aa33431f6e562bc94b81146959021575393e08e032daf469b57bde2fd8 Jun 10 17:41:13.978158 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.978132 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cvnz\" (UniqueName: \"kubernetes.io/projected/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-kube-api-access-9cvnz\") pod \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\" (UID: \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\") " Jun 10 17:41:13.978248 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.978175 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-extensions-socket-volume\") pod \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\" (UID: \"db2d4c07-8c7f-4f59-a9a1-a87500f007f0\") " Jun 10 17:41:13.978248 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.978216 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4hnf\" (UniqueName: \"kubernetes.io/projected/7e9897b3-7f71-4a36-a97c-9e32a39d4d03-kube-api-access-p4hnf\") pod \"7e9897b3-7f71-4a36-a97c-9e32a39d4d03\" (UID: \"7e9897b3-7f71-4a36-a97c-9e32a39d4d03\") " Jun 10 17:41:13.978637 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.978610 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "db2d4c07-8c7f-4f59-a9a1-a87500f007f0" (UID: "db2d4c07-8c7f-4f59-a9a1-a87500f007f0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 17:41:13.980506 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.980482 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e9897b3-7f71-4a36-a97c-9e32a39d4d03-kube-api-access-p4hnf" (OuterVolumeSpecName: "kube-api-access-p4hnf") pod "7e9897b3-7f71-4a36-a97c-9e32a39d4d03" (UID: "7e9897b3-7f71-4a36-a97c-9e32a39d4d03"). InnerVolumeSpecName "kube-api-access-p4hnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:41:13.980604 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:13.980541 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-kube-api-access-9cvnz" (OuterVolumeSpecName: "kube-api-access-9cvnz") pod "db2d4c07-8c7f-4f59-a9a1-a87500f007f0" (UID: "db2d4c07-8c7f-4f59-a9a1-a87500f007f0"). InnerVolumeSpecName "kube-api-access-9cvnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:41:14.079768 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.079673 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cvnz\" (UniqueName: \"kubernetes.io/projected/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-kube-api-access-9cvnz\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:14.079768 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.079708 2531 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db2d4c07-8c7f-4f59-a9a1-a87500f007f0-extensions-socket-volume\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:14.079768 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.079721 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4hnf\" (UniqueName: \"kubernetes.io/projected/7e9897b3-7f71-4a36-a97c-9e32a39d4d03-kube-api-access-p4hnf\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:14.469430 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.469355 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" path="/var/lib/kubelet/pods/7e9897b3-7f71-4a36-a97c-9e32a39d4d03/volumes" Jun 10 17:41:14.469707 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.469695 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" path="/var/lib/kubelet/pods/db2d4c07-8c7f-4f59-a9a1-a87500f007f0/volumes" Jun 10 17:41:14.736335 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.736252 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" event={"ID":"185b864b-5a36-44b5-89b1-eb0f58716a38","Type":"ContainerStarted","Data":"098943100c12d2fc7abfb3235ef55f190df51ff22a5c120dd95096678c1cddb4"} Jun 10 17:41:14.736335 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.736305 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" event={"ID":"185b864b-5a36-44b5-89b1-eb0f58716a38","Type":"ContainerStarted","Data":"8368d2aa33431f6e562bc94b81146959021575393e08e032daf469b57bde2fd8"} Jun 10 17:41:14.736531 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.736367 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:14.737559 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.737534 2531 generic.go:356] "Generic (PLEG): container finished" podID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" containerID="d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac" exitCode=0 Jun 10 17:41:14.737680 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.737605 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:41:14.737680 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.737616 2531 scope.go:117] "RemoveContainer" containerID="d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac" Jun 10 17:41:14.739257 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.739232 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" event={"ID":"15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd","Type":"ContainerStarted","Data":"bedd8be721d570f3e312aab8719941d70e6cb0f7d143cbfb3630487e25dc3da4"} Jun 10 17:41:14.740724 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.740703 2531 generic.go:356] "Generic (PLEG): container finished" podID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" containerID="812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0" exitCode=0 Jun 10 17:41:14.740867 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.740746 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:41:14.746283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.746267 2531 scope.go:117] "RemoveContainer" containerID="d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac" Jun 10 17:41:14.746513 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:14.746495 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac\": container with ID starting with d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac not found: ID does not exist" containerID="d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac" Jun 10 17:41:14.746561 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.746521 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac"} err="failed to get container status \"d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac\": rpc error: code = NotFound desc = could not find container \"d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac\": container with ID starting with d4b8831a5b725d2fa1677ae11fdd3a42f567889db58cbbc46be2fb7cc85606ac not found: ID does not exist" Jun 10 17:41:14.746561 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.746536 2531 scope.go:117] "RemoveContainer" containerID="812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0" Jun 10 17:41:14.753554 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.753519 2531 scope.go:117] "RemoveContainer" containerID="812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0" Jun 10 17:41:14.753793 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:14.753771 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0\": container with ID starting with 812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0 not found: ID does not exist" containerID="812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0" Jun 10 17:41:14.753877 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.753795 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0"} err="failed to get container status \"812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0\": rpc error: code = NotFound desc = could not find container \"812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0\": container with ID starting with 812db5116e0343b9e55f9bc1b8f18626d73d4f31292080c859a9c14cdbe2c9e0 not found: ID does not exist" Jun 10 17:41:14.755095 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:14.755050 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ggqp2\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="db2d4c07-8c7f-4f59-a9a1-a87500f007f0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ggqp2" Jun 10 17:41:14.755465 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.755430 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" podStartSLOduration=13.755418939 podStartE2EDuration="13.755418939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:41:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:41:14.753030115 +0000 UTC m=+584.721236155" watchObservedRunningTime="2026-06-10 17:41:14.755418939 +0000 UTC m=+584.723624977" Jun 10 17:41:14.756550 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:14.756519 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wb5jd\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wb5jd" Jun 10 17:41:14.773321 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:14.773280 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pjl5g" podStartSLOduration=2.496457122 podStartE2EDuration="24.77326901s" totalImagesPullingTime="22.276811888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:40:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:41:14.772448304 +0000 UTC m=+584.740654344" watchObservedRunningTime="2026-06-10 17:41:14.77326901 +0000 UTC m=+584.741475045" Jun 10 17:41:17.528156 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.528123 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt"] Jun 10 17:41:17.528515 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.528305 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" podUID="185b864b-5a36-44b5-89b1-eb0f58716a38" containerName="manager" containerID="cri-o://098943100c12d2fc7abfb3235ef55f190df51ff22a5c120dd95096678c1cddb4" gracePeriod=10 Jun 10 17:41:17.756868 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.756841 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs"] Jun 10 17:41:17.757183 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.757168 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" containerName="manager" Jun 10 17:41:17.757256 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.757186 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" containerName="manager" Jun 10 17:41:17.757256 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.757249 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e9897b3-7f71-4a36-a97c-9e32a39d4d03" containerName="manager" Jun 10 17:41:17.758521 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.758500 2531 generic.go:356] "Generic (PLEG): container finished" podID="185b864b-5a36-44b5-89b1-eb0f58716a38" containerID="098943100c12d2fc7abfb3235ef55f190df51ff22a5c120dd95096678c1cddb4" exitCode=0 Jun 10 17:41:17.760295 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.760273 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" event={"ID":"185b864b-5a36-44b5-89b1-eb0f58716a38","Type":"ContainerDied","Data":"098943100c12d2fc7abfb3235ef55f190df51ff22a5c120dd95096678c1cddb4"} Jun 10 17:41:17.760379 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.760366 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:17.774335 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.774311 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:17.774441 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.774407 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs"] Jun 10 17:41:17.813454 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.813377 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkqwn\" (UniqueName: \"kubernetes.io/projected/b7585638-a85e-49d9-b0ce-54c80e6c562c-kube-api-access-kkqwn\") pod \"kuadrant-operator-controller-manager-86878c769f-vdwfs\" (UID: \"b7585638-a85e-49d9-b0ce-54c80e6c562c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:17.813597 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.813492 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7585638-a85e-49d9-b0ce-54c80e6c562c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vdwfs\" (UID: \"b7585638-a85e-49d9-b0ce-54c80e6c562c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:17.914088 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.914048 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/185b864b-5a36-44b5-89b1-eb0f58716a38-extensions-socket-volume\") pod \"185b864b-5a36-44b5-89b1-eb0f58716a38\" (UID: \"185b864b-5a36-44b5-89b1-eb0f58716a38\") " Jun 10 17:41:17.914272 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.914142 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvqsc\" (UniqueName: \"kubernetes.io/projected/185b864b-5a36-44b5-89b1-eb0f58716a38-kube-api-access-wvqsc\") pod \"185b864b-5a36-44b5-89b1-eb0f58716a38\" (UID: \"185b864b-5a36-44b5-89b1-eb0f58716a38\") " Jun 10 17:41:17.914358 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.914340 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7585638-a85e-49d9-b0ce-54c80e6c562c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vdwfs\" (UID: \"b7585638-a85e-49d9-b0ce-54c80e6c562c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:17.914423 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.914410 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkqwn\" (UniqueName: \"kubernetes.io/projected/b7585638-a85e-49d9-b0ce-54c80e6c562c-kube-api-access-kkqwn\") pod \"kuadrant-operator-controller-manager-86878c769f-vdwfs\" (UID: \"b7585638-a85e-49d9-b0ce-54c80e6c562c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:17.914476 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.914420 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/185b864b-5a36-44b5-89b1-eb0f58716a38-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "185b864b-5a36-44b5-89b1-eb0f58716a38" (UID: "185b864b-5a36-44b5-89b1-eb0f58716a38"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 17:41:17.914725 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.914702 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7585638-a85e-49d9-b0ce-54c80e6c562c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vdwfs\" (UID: \"b7585638-a85e-49d9-b0ce-54c80e6c562c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:17.916409 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.916379 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/185b864b-5a36-44b5-89b1-eb0f58716a38-kube-api-access-wvqsc" (OuterVolumeSpecName: "kube-api-access-wvqsc") pod "185b864b-5a36-44b5-89b1-eb0f58716a38" (UID: "185b864b-5a36-44b5-89b1-eb0f58716a38"). InnerVolumeSpecName "kube-api-access-wvqsc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:41:17.922701 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:17.922680 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkqwn\" (UniqueName: \"kubernetes.io/projected/b7585638-a85e-49d9-b0ce-54c80e6c562c-kube-api-access-kkqwn\") pod \"kuadrant-operator-controller-manager-86878c769f-vdwfs\" (UID: \"b7585638-a85e-49d9-b0ce-54c80e6c562c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:18.015593 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.015562 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvqsc\" (UniqueName: \"kubernetes.io/projected/185b864b-5a36-44b5-89b1-eb0f58716a38-kube-api-access-wvqsc\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:18.015593 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.015587 2531 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/185b864b-5a36-44b5-89b1-eb0f58716a38-extensions-socket-volume\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:18.088457 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.088374 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:18.215111 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.215068 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs"] Jun 10 17:41:18.218791 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:41:18.218768 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7585638_a85e_49d9_b0ce_54c80e6c562c.slice/crio-4a33ffedb0e6df332362c04b01344c616ecbfc0793c72fefacddf1e97af26c93 WatchSource:0}: Error finding container 4a33ffedb0e6df332362c04b01344c616ecbfc0793c72fefacddf1e97af26c93: Status 404 returned error can't find the container with id 4a33ffedb0e6df332362c04b01344c616ecbfc0793c72fefacddf1e97af26c93 Jun 10 17:41:18.764678 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.764632 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" event={"ID":"b7585638-a85e-49d9-b0ce-54c80e6c562c","Type":"ContainerStarted","Data":"9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992"} Jun 10 17:41:18.765141 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.764801 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:18.765141 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.764865 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" event={"ID":"b7585638-a85e-49d9-b0ce-54c80e6c562c","Type":"ContainerStarted","Data":"4a33ffedb0e6df332362c04b01344c616ecbfc0793c72fefacddf1e97af26c93"} Jun 10 17:41:18.766066 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.766045 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" event={"ID":"185b864b-5a36-44b5-89b1-eb0f58716a38","Type":"ContainerDied","Data":"8368d2aa33431f6e562bc94b81146959021575393e08e032daf469b57bde2fd8"} Jun 10 17:41:18.766066 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.766060 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt" Jun 10 17:41:18.766215 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.766088 2531 scope.go:117] "RemoveContainer" containerID="098943100c12d2fc7abfb3235ef55f190df51ff22a5c120dd95096678c1cddb4" Jun 10 17:41:18.798022 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.797976 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" podStartSLOduration=1.797959322 podStartE2EDuration="1.797959322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:41:18.796590993 +0000 UTC m=+588.764797031" watchObservedRunningTime="2026-06-10 17:41:18.797959322 +0000 UTC m=+588.766165428" Jun 10 17:41:18.829800 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.829770 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt"] Jun 10 17:41:18.833927 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:18.833900 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5rdwt"] Jun 10 17:41:20.469107 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:20.469072 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="185b864b-5a36-44b5-89b1-eb0f58716a38" path="/var/lib/kubelet/pods/185b864b-5a36-44b5-89b1-eb0f58716a38/volumes" Jun 10 17:41:23.736157 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:23.736092 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5c784d7ffb-smwcx" podUID="97606067-13ac-4e6a-b5cd-9589857e9003" containerName="console" containerID="cri-o://be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b" gracePeriod=15 Jun 10 17:41:23.978092 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:23.978070 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:41:24.071728 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.071689 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-oauth-serving-cert\") pod \"97606067-13ac-4e6a-b5cd-9589857e9003\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " Jun 10 17:41:24.071931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.071741 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-serving-cert\") pod \"97606067-13ac-4e6a-b5cd-9589857e9003\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " Jun 10 17:41:24.071931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.071766 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-trusted-ca-bundle\") pod \"97606067-13ac-4e6a-b5cd-9589857e9003\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " Jun 10 17:41:24.071931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.071800 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-console-config\") pod \"97606067-13ac-4e6a-b5cd-9589857e9003\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " Jun 10 17:41:24.071931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.071870 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-service-ca\") pod \"97606067-13ac-4e6a-b5cd-9589857e9003\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " Jun 10 17:41:24.071931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.071896 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-oauth-config\") pod \"97606067-13ac-4e6a-b5cd-9589857e9003\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " Jun 10 17:41:24.071931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.071927 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drmzt\" (UniqueName: \"kubernetes.io/projected/97606067-13ac-4e6a-b5cd-9589857e9003-kube-api-access-drmzt\") pod \"97606067-13ac-4e6a-b5cd-9589857e9003\" (UID: \"97606067-13ac-4e6a-b5cd-9589857e9003\") " Jun 10 17:41:24.072259 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.072239 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "97606067-13ac-4e6a-b5cd-9589857e9003" (UID: "97606067-13ac-4e6a-b5cd-9589857e9003"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:41:24.072310 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.072274 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "97606067-13ac-4e6a-b5cd-9589857e9003" (UID: "97606067-13ac-4e6a-b5cd-9589857e9003"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:41:24.072371 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.072306 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-service-ca" (OuterVolumeSpecName: "service-ca") pod "97606067-13ac-4e6a-b5cd-9589857e9003" (UID: "97606067-13ac-4e6a-b5cd-9589857e9003"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:41:24.072371 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.072317 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-console-config" (OuterVolumeSpecName: "console-config") pod "97606067-13ac-4e6a-b5cd-9589857e9003" (UID: "97606067-13ac-4e6a-b5cd-9589857e9003"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:41:24.074148 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.074122 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "97606067-13ac-4e6a-b5cd-9589857e9003" (UID: "97606067-13ac-4e6a-b5cd-9589857e9003"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:41:24.074344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.074314 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "97606067-13ac-4e6a-b5cd-9589857e9003" (UID: "97606067-13ac-4e6a-b5cd-9589857e9003"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:41:24.074344 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.074337 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97606067-13ac-4e6a-b5cd-9589857e9003-kube-api-access-drmzt" (OuterVolumeSpecName: "kube-api-access-drmzt") pod "97606067-13ac-4e6a-b5cd-9589857e9003" (UID: "97606067-13ac-4e6a-b5cd-9589857e9003"). InnerVolumeSpecName "kube-api-access-drmzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:41:24.172860 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.172831 2531 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-oauth-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:24.172860 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.172861 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drmzt\" (UniqueName: \"kubernetes.io/projected/97606067-13ac-4e6a-b5cd-9589857e9003-kube-api-access-drmzt\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:24.173035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.172876 2531 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-oauth-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:24.173035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.172890 2531 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97606067-13ac-4e6a-b5cd-9589857e9003-console-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:24.173035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.172903 2531 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-trusted-ca-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:24.173035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.172918 2531 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-console-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:24.173035 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.172930 2531 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97606067-13ac-4e6a-b5cd-9589857e9003-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:24.793378 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.793345 2531 generic.go:356] "Generic (PLEG): container finished" podID="97606067-13ac-4e6a-b5cd-9589857e9003" containerID="be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b" exitCode=0 Jun 10 17:41:24.793753 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.793391 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c784d7ffb-smwcx" Jun 10 17:41:24.793753 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.793436 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c784d7ffb-smwcx" event={"ID":"97606067-13ac-4e6a-b5cd-9589857e9003","Type":"ContainerDied","Data":"be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b"} Jun 10 17:41:24.793753 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.793491 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c784d7ffb-smwcx" event={"ID":"97606067-13ac-4e6a-b5cd-9589857e9003","Type":"ContainerDied","Data":"e1dfe7cb7bd6c5ba365bc131f3ebfa18f4d72e2a0cb658fbabdd31ce7cd4e1e4"} Jun 10 17:41:24.793753 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.793518 2531 scope.go:117] "RemoveContainer" containerID="be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b" Jun 10 17:41:24.801698 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.801681 2531 scope.go:117] "RemoveContainer" containerID="be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b" Jun 10 17:41:24.802027 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:24.801915 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b\": container with ID starting with be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b not found: ID does not exist" containerID="be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b" Jun 10 17:41:24.802027 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.801967 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b"} err="failed to get container status \"be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b\": rpc error: code = NotFound desc = could not find container \"be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b\": container with ID starting with be2431f46ff61247a25d34368085abb11669b379e35738011bb17733ba66b62b not found: ID does not exist" Jun 10 17:41:24.808190 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.808162 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c784d7ffb-smwcx"] Jun 10 17:41:24.812183 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:24.812165 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c784d7ffb-smwcx"] Jun 10 17:41:26.473557 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:26.473520 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97606067-13ac-4e6a-b5cd-9589857e9003" path="/var/lib/kubelet/pods/97606067-13ac-4e6a-b5cd-9589857e9003/volumes" Jun 10 17:41:29.773750 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:29.773718 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 17:41:30.433402 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:30.433373 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:41:30.434326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:30.434304 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:41:33.800873 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.800840 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67"] Jun 10 17:41:33.801358 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.801161 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="185b864b-5a36-44b5-89b1-eb0f58716a38" containerName="manager" Jun 10 17:41:33.801358 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.801174 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="185b864b-5a36-44b5-89b1-eb0f58716a38" containerName="manager" Jun 10 17:41:33.801358 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.801195 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97606067-13ac-4e6a-b5cd-9589857e9003" containerName="console" Jun 10 17:41:33.801358 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.801200 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="97606067-13ac-4e6a-b5cd-9589857e9003" containerName="console" Jun 10 17:41:33.801358 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.801257 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="185b864b-5a36-44b5-89b1-eb0f58716a38" containerName="manager" Jun 10 17:41:33.801358 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.801267 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="97606067-13ac-4e6a-b5cd-9589857e9003" containerName="console" Jun 10 17:41:33.942846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.942808 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67"] Jun 10 17:41:33.943004 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.942903 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:33.945154 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:33.945130 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4kcv4\"" Jun 10 17:41:34.054476 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054412 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.054476 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054453 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.054476 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054473 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.054663 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054517 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.054663 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054572 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.054663 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054589 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a533ebf2-5111-41c2-8322-c7a463e38275-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.054663 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054622 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7n8z\" (UniqueName: \"kubernetes.io/projected/a533ebf2-5111-41c2-8322-c7a463e38275-kube-api-access-k7n8z\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.054781 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054678 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a533ebf2-5111-41c2-8322-c7a463e38275-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.054781 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.054734 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a533ebf2-5111-41c2-8322-c7a463e38275-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156099 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156063 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156099 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156106 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156124 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156145 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156174 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156202 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a533ebf2-5111-41c2-8322-c7a463e38275-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156238 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7n8z\" (UniqueName: \"kubernetes.io/projected/a533ebf2-5111-41c2-8322-c7a463e38275-kube-api-access-k7n8z\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156268 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a533ebf2-5111-41c2-8322-c7a463e38275-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156572 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156327 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a533ebf2-5111-41c2-8322-c7a463e38275-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156572 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156514 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156675 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156602 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156675 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156642 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.156852 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.156835 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.157269 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.157250 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a533ebf2-5111-41c2-8322-c7a463e38275-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.158666 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.158644 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a533ebf2-5111-41c2-8322-c7a463e38275-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.158739 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.158709 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a533ebf2-5111-41c2-8322-c7a463e38275-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.163450 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.163430 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a533ebf2-5111-41c2-8322-c7a463e38275-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.163792 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.163769 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7n8z\" (UniqueName: \"kubernetes.io/projected/a533ebf2-5111-41c2-8322-c7a463e38275-kube-api-access-k7n8z\") pod \"maas-default-gateway-openshift-default-687ff6996-98d67\" (UID: \"a533ebf2-5111-41c2-8322-c7a463e38275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.253258 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.253226 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:34.410347 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.410230 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67"] Jun 10 17:41:34.412962 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:41:34.412916 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda533ebf2_5111_41c2_8322_c7a463e38275.slice/crio-53d536abbf6012c413371dbabadc4aa2f3a429fefee3d22e74d1012bc155c0cf WatchSource:0}: Error finding container 53d536abbf6012c413371dbabadc4aa2f3a429fefee3d22e74d1012bc155c0cf: Status 404 returned error can't find the container with id 53d536abbf6012c413371dbabadc4aa2f3a429fefee3d22e74d1012bc155c0cf Jun 10 17:41:34.415231 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.415202 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:41:34.415325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.415263 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:41:34.415325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.415295 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:41:34.845702 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.845655 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" event={"ID":"a533ebf2-5111-41c2-8322-c7a463e38275","Type":"ContainerStarted","Data":"627f7cdf112d08681a0b26ac59f6fdcbb14d76b8b512f6279624ca66418ce09f"} Jun 10 17:41:34.846108 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.845746 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" event={"ID":"a533ebf2-5111-41c2-8322-c7a463e38275","Type":"ContainerStarted","Data":"53d536abbf6012c413371dbabadc4aa2f3a429fefee3d22e74d1012bc155c0cf"} Jun 10 17:41:34.863075 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:34.863024 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" podStartSLOduration=1.863009471 podStartE2EDuration="1.863009471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:41:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:41:34.86261825 +0000 UTC m=+604.830824288" watchObservedRunningTime="2026-06-10 17:41:34.863009471 +0000 UTC m=+604.831215510" Jun 10 17:41:35.254432 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:35.254404 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:35.259341 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:35.259316 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:35.850554 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:35.850523 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:35.851419 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:35.851399 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:41:48.498125 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.498050 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fkfq"] Jun 10 17:41:48.501608 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.501593 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:48.504145 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.504128 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 10 17:41:48.509498 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.509477 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fkfq"] Jun 10 17:41:48.569742 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.569706 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9d9\" (UniqueName: \"kubernetes.io/projected/1f4b345c-ec16-4e79-97f8-73a353214099-kube-api-access-nw9d9\") pod \"limitador-limitador-64c94fd64f-6fkfq\" (UID: \"1f4b345c-ec16-4e79-97f8-73a353214099\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:48.569886 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.569756 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1f4b345c-ec16-4e79-97f8-73a353214099-config-file\") pod \"limitador-limitador-64c94fd64f-6fkfq\" (UID: \"1f4b345c-ec16-4e79-97f8-73a353214099\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:48.594515 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.594488 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fkfq"] Jun 10 17:41:48.670576 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.670542 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9d9\" (UniqueName: \"kubernetes.io/projected/1f4b345c-ec16-4e79-97f8-73a353214099-kube-api-access-nw9d9\") pod \"limitador-limitador-64c94fd64f-6fkfq\" (UID: \"1f4b345c-ec16-4e79-97f8-73a353214099\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:48.670576 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.670579 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1f4b345c-ec16-4e79-97f8-73a353214099-config-file\") pod \"limitador-limitador-64c94fd64f-6fkfq\" (UID: \"1f4b345c-ec16-4e79-97f8-73a353214099\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:48.671178 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.671161 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1f4b345c-ec16-4e79-97f8-73a353214099-config-file\") pod \"limitador-limitador-64c94fd64f-6fkfq\" (UID: \"1f4b345c-ec16-4e79-97f8-73a353214099\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:48.679319 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.679293 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9d9\" (UniqueName: \"kubernetes.io/projected/1f4b345c-ec16-4e79-97f8-73a353214099-kube-api-access-nw9d9\") pod \"limitador-limitador-64c94fd64f-6fkfq\" (UID: \"1f4b345c-ec16-4e79-97f8-73a353214099\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:48.812603 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.812531 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:48.956504 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:48.956480 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fkfq"] Jun 10 17:41:48.960898 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:41:48.960867 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4b345c_ec16_4e79_97f8_73a353214099.slice/crio-145394026e02ca7be4921220b8dc9e91e2b851ff95794cddcd2988898d791fb3 WatchSource:0}: Error finding container 145394026e02ca7be4921220b8dc9e91e2b851ff95794cddcd2988898d791fb3: Status 404 returned error can't find the container with id 145394026e02ca7be4921220b8dc9e91e2b851ff95794cddcd2988898d791fb3 Jun 10 17:41:49.306550 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.306516 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-t4xvl"] Jun 10 17:41:49.308687 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.308671 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-t4xvl" Jun 10 17:41:49.310740 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.310722 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zkhq5\"" Jun 10 17:41:49.317286 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.317266 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-t4xvl"] Jun 10 17:41:49.374473 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.374445 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql8tv\" (UniqueName: \"kubernetes.io/projected/1cc756a1-00ca-49ac-b359-9c9756a9674c-kube-api-access-ql8tv\") pod \"authorino-58cd955675-t4xvl\" (UID: \"1cc756a1-00ca-49ac-b359-9c9756a9674c\") " pod="kuadrant-system/authorino-58cd955675-t4xvl" Jun 10 17:41:49.475128 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.475097 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql8tv\" (UniqueName: \"kubernetes.io/projected/1cc756a1-00ca-49ac-b359-9c9756a9674c-kube-api-access-ql8tv\") pod \"authorino-58cd955675-t4xvl\" (UID: \"1cc756a1-00ca-49ac-b359-9c9756a9674c\") " pod="kuadrant-system/authorino-58cd955675-t4xvl" Jun 10 17:41:49.483483 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.483461 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql8tv\" (UniqueName: \"kubernetes.io/projected/1cc756a1-00ca-49ac-b359-9c9756a9674c-kube-api-access-ql8tv\") pod \"authorino-58cd955675-t4xvl\" (UID: \"1cc756a1-00ca-49ac-b359-9c9756a9674c\") " pod="kuadrant-system/authorino-58cd955675-t4xvl" Jun 10 17:41:49.618721 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.618638 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-t4xvl" Jun 10 17:41:49.772389 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.771987 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-t4xvl"] Jun 10 17:41:49.776060 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:41:49.776002 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cc756a1_00ca_49ac_b359_9c9756a9674c.slice/crio-a0f5564922c556f9dcd7a82f971f83f9a5e2dddd6496b07d1aa1727eec6b3c24 WatchSource:0}: Error finding container a0f5564922c556f9dcd7a82f971f83f9a5e2dddd6496b07d1aa1727eec6b3c24: Status 404 returned error can't find the container with id a0f5564922c556f9dcd7a82f971f83f9a5e2dddd6496b07d1aa1727eec6b3c24 Jun 10 17:41:49.924765 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.924645 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" event={"ID":"1f4b345c-ec16-4e79-97f8-73a353214099","Type":"ContainerStarted","Data":"145394026e02ca7be4921220b8dc9e91e2b851ff95794cddcd2988898d791fb3"} Jun 10 17:41:49.925859 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:49.925826 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-t4xvl" event={"ID":"1cc756a1-00ca-49ac-b359-9c9756a9674c","Type":"ContainerStarted","Data":"a0f5564922c556f9dcd7a82f971f83f9a5e2dddd6496b07d1aa1727eec6b3c24"} Jun 10 17:41:52.943248 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:52.943210 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" event={"ID":"1f4b345c-ec16-4e79-97f8-73a353214099","Type":"ContainerStarted","Data":"475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8"} Jun 10 17:41:52.943659 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:52.943376 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:41:52.958930 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:52.958877 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" podStartSLOduration=2.028050442 podStartE2EDuration="4.958862513s" totalImagesPullingTime="2.930812071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:41:52.957223492 +0000 UTC m=+622.925429531" watchObservedRunningTime="2026-06-10 17:41:52.958862513 +0000 UTC m=+622.927068554" Jun 10 17:41:53.949111 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:53.949012 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-t4xvl" event={"ID":"1cc756a1-00ca-49ac-b359-9c9756a9674c","Type":"ContainerStarted","Data":"2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e"} Jun 10 17:41:53.962427 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:53.962370 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-t4xvl" podStartSLOduration=1.124726245 podStartE2EDuration="4.962354779s" totalImagesPullingTime="3.837628534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:41:53.960780526 +0000 UTC m=+623.928986543" watchObservedRunningTime="2026-06-10 17:41:53.962354779 +0000 UTC m=+623.930560822" Jun 10 17:41:54.762258 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:54.762223 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-t4xvl"] Jun 10 17:41:55.956845 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:55.956786 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-t4xvl" podUID="1cc756a1-00ca-49ac-b359-9c9756a9674c" containerName="authorino" containerID="cri-o://2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e" gracePeriod=30 Jun 10 17:41:56.205563 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.205542 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-t4xvl" Jun 10 17:41:56.231819 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.231766 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql8tv\" (UniqueName: \"kubernetes.io/projected/1cc756a1-00ca-49ac-b359-9c9756a9674c-kube-api-access-ql8tv\") pod \"1cc756a1-00ca-49ac-b359-9c9756a9674c\" (UID: \"1cc756a1-00ca-49ac-b359-9c9756a9674c\") " Jun 10 17:41:56.233844 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.233815 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc756a1-00ca-49ac-b359-9c9756a9674c-kube-api-access-ql8tv" (OuterVolumeSpecName: "kube-api-access-ql8tv") pod "1cc756a1-00ca-49ac-b359-9c9756a9674c" (UID: "1cc756a1-00ca-49ac-b359-9c9756a9674c"). InnerVolumeSpecName "kube-api-access-ql8tv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:41:56.332624 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.332600 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql8tv\" (UniqueName: \"kubernetes.io/projected/1cc756a1-00ca-49ac-b359-9c9756a9674c-kube-api-access-ql8tv\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:41:56.961861 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.961828 2531 generic.go:356] "Generic (PLEG): container finished" podID="1cc756a1-00ca-49ac-b359-9c9756a9674c" containerID="2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e" exitCode=0 Jun 10 17:41:56.961861 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.961867 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-t4xvl" event={"ID":"1cc756a1-00ca-49ac-b359-9c9756a9674c","Type":"ContainerDied","Data":"2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e"} Jun 10 17:41:56.962402 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.961877 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-t4xvl" Jun 10 17:41:56.962402 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.961895 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-t4xvl" event={"ID":"1cc756a1-00ca-49ac-b359-9c9756a9674c","Type":"ContainerDied","Data":"a0f5564922c556f9dcd7a82f971f83f9a5e2dddd6496b07d1aa1727eec6b3c24"} Jun 10 17:41:56.962402 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.961912 2531 scope.go:117] "RemoveContainer" containerID="2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e" Jun 10 17:41:56.970148 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.970127 2531 scope.go:117] "RemoveContainer" containerID="2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e" Jun 10 17:41:56.970374 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:41:56.970356 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e\": container with ID starting with 2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e not found: ID does not exist" containerID="2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e" Jun 10 17:41:56.970428 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.970382 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e"} err="failed to get container status \"2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e\": rpc error: code = NotFound desc = could not find container \"2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e\": container with ID starting with 2e135dac5f00f1a8eb4a4642b571b54aa912fe3d94a73f43a86a56aac1372f5e not found: ID does not exist" Jun 10 17:41:56.976568 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.976547 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-t4xvl"] Jun 10 17:41:56.981863 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:56.981839 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-t4xvl"] Jun 10 17:41:58.469856 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:41:58.469821 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc756a1-00ca-49ac-b359-9c9756a9674c" path="/var/lib/kubelet/pods/1cc756a1-00ca-49ac-b359-9c9756a9674c/volumes" Jun 10 17:42:03.950234 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:03.950204 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:42:05.780504 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:05.780469 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fkfq"] Jun 10 17:42:05.780913 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:05.780677 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" podUID="1f4b345c-ec16-4e79-97f8-73a353214099" containerName="limitador" containerID="cri-o://475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8" gracePeriod=30 Jun 10 17:42:06.320204 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:06.320181 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:42:06.413143 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:06.413078 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw9d9\" (UniqueName: \"kubernetes.io/projected/1f4b345c-ec16-4e79-97f8-73a353214099-kube-api-access-nw9d9\") pod \"1f4b345c-ec16-4e79-97f8-73a353214099\" (UID: \"1f4b345c-ec16-4e79-97f8-73a353214099\") " Jun 10 17:42:06.413281 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:06.413149 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1f4b345c-ec16-4e79-97f8-73a353214099-config-file\") pod \"1f4b345c-ec16-4e79-97f8-73a353214099\" (UID: \"1f4b345c-ec16-4e79-97f8-73a353214099\") " Jun 10 17:42:06.413510 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:06.413489 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4b345c-ec16-4e79-97f8-73a353214099-config-file" (OuterVolumeSpecName: "config-file") pod "1f4b345c-ec16-4e79-97f8-73a353214099" (UID: "1f4b345c-ec16-4e79-97f8-73a353214099"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:42:06.415222 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:06.415199 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4b345c-ec16-4e79-97f8-73a353214099-kube-api-access-nw9d9" (OuterVolumeSpecName: "kube-api-access-nw9d9") pod "1f4b345c-ec16-4e79-97f8-73a353214099" (UID: "1f4b345c-ec16-4e79-97f8-73a353214099"). InnerVolumeSpecName "kube-api-access-nw9d9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:42:06.514624 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:06.514588 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw9d9\" (UniqueName: \"kubernetes.io/projected/1f4b345c-ec16-4e79-97f8-73a353214099-kube-api-access-nw9d9\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:42:06.514624 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:06.514624 2531 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1f4b345c-ec16-4e79-97f8-73a353214099-config-file\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:42:07.004398 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.004360 2531 generic.go:356] "Generic (PLEG): container finished" podID="1f4b345c-ec16-4e79-97f8-73a353214099" containerID="475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8" exitCode=0 Jun 10 17:42:07.004846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.004414 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" Jun 10 17:42:07.004846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.004449 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" event={"ID":"1f4b345c-ec16-4e79-97f8-73a353214099","Type":"ContainerDied","Data":"475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8"} Jun 10 17:42:07.004846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.004514 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fkfq" event={"ID":"1f4b345c-ec16-4e79-97f8-73a353214099","Type":"ContainerDied","Data":"145394026e02ca7be4921220b8dc9e91e2b851ff95794cddcd2988898d791fb3"} Jun 10 17:42:07.004846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.004550 2531 scope.go:117] "RemoveContainer" containerID="475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8" Jun 10 17:42:07.012705 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.012689 2531 scope.go:117] "RemoveContainer" containerID="475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8" Jun 10 17:42:07.012967 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:42:07.012930 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8\": container with ID starting with 475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8 not found: ID does not exist" containerID="475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8" Jun 10 17:42:07.013022 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.012975 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8"} err="failed to get container status \"475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8\": rpc error: code = NotFound desc = could not find container \"475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8\": container with ID starting with 475027e66c3977d1984ebd54df56aa2d0ee69240d0be91af37a7106e711562e8 not found: ID does not exist" Jun 10 17:42:07.018681 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.018661 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fkfq"] Jun 10 17:42:07.022562 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:07.022542 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fkfq"] Jun 10 17:42:08.468851 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:08.468814 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f4b345c-ec16-4e79-97f8-73a353214099" path="/var/lib/kubelet/pods/1f4b345c-ec16-4e79-97f8-73a353214099/volumes" Jun 10 17:42:09.354743 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.354711 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-zpn48"] Jun 10 17:42:09.355161 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.355137 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f4b345c-ec16-4e79-97f8-73a353214099" containerName="limitador" Jun 10 17:42:09.355161 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.355164 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f4b345c-ec16-4e79-97f8-73a353214099" containerName="limitador" Jun 10 17:42:09.355337 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.355176 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc756a1-00ca-49ac-b359-9c9756a9674c" containerName="authorino" Jun 10 17:42:09.355337 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.355184 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc756a1-00ca-49ac-b359-9c9756a9674c" containerName="authorino" Jun 10 17:42:09.355337 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.355264 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f4b345c-ec16-4e79-97f8-73a353214099" containerName="limitador" Jun 10 17:42:09.355337 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.355274 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc756a1-00ca-49ac-b359-9c9756a9674c" containerName="authorino" Jun 10 17:42:09.357829 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.357806 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:09.359700 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.359681 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 10 17:42:09.364443 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.364422 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-zpn48"] Jun 10 17:42:09.437699 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.437674 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f4d99a9-9336-45a9-b4cd-899ed1e3477d-data\") pod \"postgres-69fb4594cf-zpn48\" (UID: \"5f4d99a9-9336-45a9-b4cd-899ed1e3477d\") " pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:09.437812 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.437724 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wstc5\" (UniqueName: \"kubernetes.io/projected/5f4d99a9-9336-45a9-b4cd-899ed1e3477d-kube-api-access-wstc5\") pod \"postgres-69fb4594cf-zpn48\" (UID: \"5f4d99a9-9336-45a9-b4cd-899ed1e3477d\") " pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:09.538724 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.538690 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wstc5\" (UniqueName: \"kubernetes.io/projected/5f4d99a9-9336-45a9-b4cd-899ed1e3477d-kube-api-access-wstc5\") pod \"postgres-69fb4594cf-zpn48\" (UID: \"5f4d99a9-9336-45a9-b4cd-899ed1e3477d\") " pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:09.539081 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.538747 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f4d99a9-9336-45a9-b4cd-899ed1e3477d-data\") pod \"postgres-69fb4594cf-zpn48\" (UID: \"5f4d99a9-9336-45a9-b4cd-899ed1e3477d\") " pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:09.539081 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.539068 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f4d99a9-9336-45a9-b4cd-899ed1e3477d-data\") pod \"postgres-69fb4594cf-zpn48\" (UID: \"5f4d99a9-9336-45a9-b4cd-899ed1e3477d\") " pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:09.546020 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.545998 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wstc5\" (UniqueName: \"kubernetes.io/projected/5f4d99a9-9336-45a9-b4cd-899ed1e3477d-kube-api-access-wstc5\") pod \"postgres-69fb4594cf-zpn48\" (UID: \"5f4d99a9-9336-45a9-b4cd-899ed1e3477d\") " pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:09.670086 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.670011 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:09.795477 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:09.795454 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-zpn48"] Jun 10 17:42:09.797372 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:42:09.797348 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4d99a9_9336_45a9_b4cd_899ed1e3477d.slice/crio-8ae323f6b764c481dbbaa4b7364d4595ab1121dc71093a0fe5ac20832965fe3c WatchSource:0}: Error finding container 8ae323f6b764c481dbbaa4b7364d4595ab1121dc71093a0fe5ac20832965fe3c: Status 404 returned error can't find the container with id 8ae323f6b764c481dbbaa4b7364d4595ab1121dc71093a0fe5ac20832965fe3c Jun 10 17:42:10.017765 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:10.017736 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-zpn48" event={"ID":"5f4d99a9-9336-45a9-b4cd-899ed1e3477d","Type":"ContainerStarted","Data":"8ae323f6b764c481dbbaa4b7364d4595ab1121dc71093a0fe5ac20832965fe3c"} Jun 10 17:42:12.028760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:12.028721 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-zpn48" event={"ID":"5f4d99a9-9336-45a9-b4cd-899ed1e3477d","Type":"ContainerStarted","Data":"61d5dba5c0cf4e33126339cae3a518f7258f1bab94cd6d859316d27e16a393e7"} Jun 10 17:42:12.029107 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:12.028825 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:12.044105 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:12.044063 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-zpn48" podStartSLOduration=0.888844659 podStartE2EDuration="3.044047705s" totalImagesPullingTime="2.155203046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:42:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:42:12.041969221 +0000 UTC m=+642.010175257" watchObservedRunningTime="2026-06-10 17:42:12.044047705 +0000 UTC m=+642.012253743" Jun 10 17:42:18.064589 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:18.064554 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-zpn48" Jun 10 17:42:42.581931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.581903 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:42:42.589349 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.589329 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 10 17:42:42.591782 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.591754 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 10 17:42:42.592356 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.592333 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 10 17:42:42.592521 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.592354 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-6zd5n\"" Jun 10 17:42:42.592667 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.592444 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 10 17:42:42.594105 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.594084 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:42:42.613977 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.613951 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9ksx\" (UniqueName: \"kubernetes.io/projected/7dbb1199-37de-4be0-8687-09c5d6e7dfa6-kube-api-access-w9ksx\") pod \"maas-keycloak-0\" (UID: \"7dbb1199-37de-4be0-8687-09c5d6e7dfa6\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:42:42.714677 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.714642 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9ksx\" (UniqueName: \"kubernetes.io/projected/7dbb1199-37de-4be0-8687-09c5d6e7dfa6-kube-api-access-w9ksx\") pod \"maas-keycloak-0\" (UID: \"7dbb1199-37de-4be0-8687-09c5d6e7dfa6\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:42:42.722409 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.722387 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9ksx\" (UniqueName: \"kubernetes.io/projected/7dbb1199-37de-4be0-8687-09c5d6e7dfa6-kube-api-access-w9ksx\") pod \"maas-keycloak-0\" (UID: \"7dbb1199-37de-4be0-8687-09c5d6e7dfa6\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:42:42.899963 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:42.899874 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 10 17:42:43.027325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:43.027302 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:42:43.029233 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:42:43.029196 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbb1199_37de_4be0_8687_09c5d6e7dfa6.slice/crio-d4bff0d2ae30025176009e5941602eebb37053649897db38d74b2a2115a77e64 WatchSource:0}: Error finding container d4bff0d2ae30025176009e5941602eebb37053649897db38d74b2a2115a77e64: Status 404 returned error can't find the container with id d4bff0d2ae30025176009e5941602eebb37053649897db38d74b2a2115a77e64 Jun 10 17:42:43.172837 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:43.172767 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7dbb1199-37de-4be0-8687-09c5d6e7dfa6","Type":"ContainerStarted","Data":"d4bff0d2ae30025176009e5941602eebb37053649897db38d74b2a2115a77e64"} Jun 10 17:42:54.232602 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:54.232563 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7dbb1199-37de-4be0-8687-09c5d6e7dfa6","Type":"ContainerStarted","Data":"8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c"} Jun 10 17:42:54.255673 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:54.255623 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.679029083 podStartE2EDuration="12.25560815s" totalImagesPullingTime="10.576579067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:42:54.254027104 +0000 UTC m=+684.222233143" watchObservedRunningTime="2026-06-10 17:42:54.25560815 +0000 UTC m=+684.223814188" Jun 10 17:42:54.900165 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:54.900123 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 10 17:42:54.902142 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:54.902103 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:42:55.901043 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:55.900996 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:42:56.900744 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:56.900686 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:42:57.900582 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:57.900538 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:42:58.900813 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:58.900764 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:42:59.901251 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:42:59.901200 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:43:00.901354 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:00.901295 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:43:01.900661 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:01.900618 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:43:02.901609 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:02.900977 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 10 17:43:02.901609 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:02.901328 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:43:03.901382 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:03.901331 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:43:04.900990 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:04.900908 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:43:05.901221 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:05.901176 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.34:9000/health/started\": dial tcp 10.134.0.34:9000: connect: connection refused" Jun 10 17:43:07.021118 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:07.021078 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 10 17:43:07.039129 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:07.039091 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 10 17:43:17.027887 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:17.027849 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 10 17:43:17.918439 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:17.918359 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xkjgm"] Jun 10 17:43:17.925926 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:17.925903 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" Jun 10 17:43:17.927947 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:17.927913 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zkhq5\"" Jun 10 17:43:17.935048 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:17.935026 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xkjgm"] Jun 10 17:43:18.036390 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.036355 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qf59\" (UniqueName: \"kubernetes.io/projected/89df2eb9-03ae-46ec-b945-cc0ae5d35c17-kube-api-access-4qf59\") pod \"authorino-5d4b67c446-xkjgm\" (UID: \"89df2eb9-03ae-46ec-b945-cc0ae5d35c17\") " pod="kuadrant-system/authorino-5d4b67c446-xkjgm" Jun 10 17:43:18.137219 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.137182 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qf59\" (UniqueName: \"kubernetes.io/projected/89df2eb9-03ae-46ec-b945-cc0ae5d35c17-kube-api-access-4qf59\") pod \"authorino-5d4b67c446-xkjgm\" (UID: \"89df2eb9-03ae-46ec-b945-cc0ae5d35c17\") " pod="kuadrant-system/authorino-5d4b67c446-xkjgm" Jun 10 17:43:18.146586 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.146563 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qf59\" (UniqueName: \"kubernetes.io/projected/89df2eb9-03ae-46ec-b945-cc0ae5d35c17-kube-api-access-4qf59\") pod \"authorino-5d4b67c446-xkjgm\" (UID: \"89df2eb9-03ae-46ec-b945-cc0ae5d35c17\") " pod="kuadrant-system/authorino-5d4b67c446-xkjgm" Jun 10 17:43:18.149993 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.149966 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xkjgm"] Jun 10 17:43:18.150206 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.150194 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" Jun 10 17:43:18.174808 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.174744 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8594bb864b-qcwt9"] Jun 10 17:43:18.177193 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.177173 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8594bb864b-qcwt9" Jun 10 17:43:18.185694 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.185669 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8594bb864b-qcwt9"] Jun 10 17:43:18.281674 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.281617 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xkjgm"] Jun 10 17:43:18.285182 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:43:18.285153 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89df2eb9_03ae_46ec_b945_cc0ae5d35c17.slice/crio-aa32cafc195f0f29478eb218d048e1e8563f856700228b704308e308d16d8486 WatchSource:0}: Error finding container aa32cafc195f0f29478eb218d048e1e8563f856700228b704308e308d16d8486: Status 404 returned error can't find the container with id aa32cafc195f0f29478eb218d048e1e8563f856700228b704308e308d16d8486 Jun 10 17:43:18.339269 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.339237 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgd4h\" (UniqueName: \"kubernetes.io/projected/28e98d38-5855-4431-a5b1-d2772e495577-kube-api-access-sgd4h\") pod \"authorino-8594bb864b-qcwt9\" (UID: \"28e98d38-5855-4431-a5b1-d2772e495577\") " pod="kuadrant-system/authorino-8594bb864b-qcwt9" Jun 10 17:43:18.367702 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.367669 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" event={"ID":"89df2eb9-03ae-46ec-b945-cc0ae5d35c17","Type":"ContainerStarted","Data":"aa32cafc195f0f29478eb218d048e1e8563f856700228b704308e308d16d8486"} Jun 10 17:43:18.440365 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.440281 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgd4h\" (UniqueName: \"kubernetes.io/projected/28e98d38-5855-4431-a5b1-d2772e495577-kube-api-access-sgd4h\") pod \"authorino-8594bb864b-qcwt9\" (UID: \"28e98d38-5855-4431-a5b1-d2772e495577\") " pod="kuadrant-system/authorino-8594bb864b-qcwt9" Jun 10 17:43:18.448347 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.448319 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgd4h\" (UniqueName: \"kubernetes.io/projected/28e98d38-5855-4431-a5b1-d2772e495577-kube-api-access-sgd4h\") pod \"authorino-8594bb864b-qcwt9\" (UID: \"28e98d38-5855-4431-a5b1-d2772e495577\") " pod="kuadrant-system/authorino-8594bb864b-qcwt9" Jun 10 17:43:18.486826 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.486798 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8594bb864b-qcwt9" Jun 10 17:43:18.524103 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.524034 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8594bb864b-qcwt9"] Jun 10 17:43:18.549812 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.549783 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74cf5dc58b-gtpgn"] Jun 10 17:43:18.552677 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.552655 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:43:18.554665 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.554643 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 10 17:43:18.562349 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.562330 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74cf5dc58b-gtpgn"] Jun 10 17:43:18.616950 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.616912 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8594bb864b-qcwt9"] Jun 10 17:43:18.620459 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:43:18.620431 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28e98d38_5855_4431_a5b1_d2772e495577.slice/crio-200bfb7f077e8540781e5711ce4065dab2c87ec367e09ef0d610d50ae3eca945 WatchSource:0}: Error finding container 200bfb7f077e8540781e5711ce4065dab2c87ec367e09ef0d610d50ae3eca945: Status 404 returned error can't find the container with id 200bfb7f077e8540781e5711ce4065dab2c87ec367e09ef0d610d50ae3eca945 Jun 10 17:43:18.642192 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.642171 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lwr8\" (UniqueName: \"kubernetes.io/projected/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-kube-api-access-7lwr8\") pod \"authorino-74cf5dc58b-gtpgn\" (UID: \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\") " pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:43:18.642286 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.642236 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-tls-cert\") pod \"authorino-74cf5dc58b-gtpgn\" (UID: \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\") " pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:43:18.743702 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.743602 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-tls-cert\") pod \"authorino-74cf5dc58b-gtpgn\" (UID: \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\") " pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:43:18.743702 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.743664 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lwr8\" (UniqueName: \"kubernetes.io/projected/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-kube-api-access-7lwr8\") pod \"authorino-74cf5dc58b-gtpgn\" (UID: \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\") " pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:43:18.746068 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.746044 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-tls-cert\") pod \"authorino-74cf5dc58b-gtpgn\" (UID: \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\") " pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:43:18.751370 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.751351 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lwr8\" (UniqueName: \"kubernetes.io/projected/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-kube-api-access-7lwr8\") pod \"authorino-74cf5dc58b-gtpgn\" (UID: \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\") " pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:43:18.862625 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.862579 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:43:18.997687 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:18.997613 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74cf5dc58b-gtpgn"] Jun 10 17:43:19.002094 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:43:19.002060 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1312fcf_04b2_4fe2_a941_2e2a9a4794e9.slice/crio-6ef015cbf669fa2c8e8b71a34a0674e1b6d799eb5b3528c657433af516fedb58 WatchSource:0}: Error finding container 6ef015cbf669fa2c8e8b71a34a0674e1b6d799eb5b3528c657433af516fedb58: Status 404 returned error can't find the container with id 6ef015cbf669fa2c8e8b71a34a0674e1b6d799eb5b3528c657433af516fedb58 Jun 10 17:43:19.375613 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.375586 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" event={"ID":"89df2eb9-03ae-46ec-b945-cc0ae5d35c17","Type":"ContainerStarted","Data":"38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec"} Jun 10 17:43:19.375919 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.375638 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" podUID="89df2eb9-03ae-46ec-b945-cc0ae5d35c17" containerName="authorino" containerID="cri-o://38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec" gracePeriod=30 Jun 10 17:43:19.377065 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.377045 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8594bb864b-qcwt9" event={"ID":"28e98d38-5855-4431-a5b1-d2772e495577","Type":"ContainerStarted","Data":"fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71"} Jun 10 17:43:19.377162 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.377073 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8594bb864b-qcwt9" event={"ID":"28e98d38-5855-4431-a5b1-d2772e495577","Type":"ContainerStarted","Data":"200bfb7f077e8540781e5711ce4065dab2c87ec367e09ef0d610d50ae3eca945"} Jun 10 17:43:19.377162 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.377081 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-8594bb864b-qcwt9" podUID="28e98d38-5855-4431-a5b1-d2772e495577" containerName="authorino" containerID="cri-o://fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71" gracePeriod=30 Jun 10 17:43:19.378356 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.378292 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" event={"ID":"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9","Type":"ContainerStarted","Data":"6ef015cbf669fa2c8e8b71a34a0674e1b6d799eb5b3528c657433af516fedb58"} Jun 10 17:43:19.392280 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.390704 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" podStartSLOduration=2.055482642 podStartE2EDuration="2.390659053s" totalImagesPullingTime="335.176411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:43:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:43:19.389141078 +0000 UTC m=+709.357347118" watchObservedRunningTime="2026-06-10 17:43:19.390659053 +0000 UTC m=+709.358865094" Jun 10 17:43:19.403988 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.403918 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8594bb864b-qcwt9" podStartSLOduration=1.139173908 podStartE2EDuration="1.403906594s" totalImagesPullingTime="264.732686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:43:19.402991528 +0000 UTC m=+709.371197567" watchObservedRunningTime="2026-06-10 17:43:19.403906594 +0000 UTC m=+709.372112633" Jun 10 17:43:19.635329 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.635304 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" Jun 10 17:43:19.647030 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.647011 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8594bb864b-qcwt9" Jun 10 17:43:19.752365 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.752329 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgd4h\" (UniqueName: \"kubernetes.io/projected/28e98d38-5855-4431-a5b1-d2772e495577-kube-api-access-sgd4h\") pod \"28e98d38-5855-4431-a5b1-d2772e495577\" (UID: \"28e98d38-5855-4431-a5b1-d2772e495577\") " Jun 10 17:43:19.752365 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.752366 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qf59\" (UniqueName: \"kubernetes.io/projected/89df2eb9-03ae-46ec-b945-cc0ae5d35c17-kube-api-access-4qf59\") pod \"89df2eb9-03ae-46ec-b945-cc0ae5d35c17\" (UID: \"89df2eb9-03ae-46ec-b945-cc0ae5d35c17\") " Jun 10 17:43:19.754316 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.754287 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e98d38-5855-4431-a5b1-d2772e495577-kube-api-access-sgd4h" (OuterVolumeSpecName: "kube-api-access-sgd4h") pod "28e98d38-5855-4431-a5b1-d2772e495577" (UID: "28e98d38-5855-4431-a5b1-d2772e495577"). InnerVolumeSpecName "kube-api-access-sgd4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:43:19.754397 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.754379 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89df2eb9-03ae-46ec-b945-cc0ae5d35c17-kube-api-access-4qf59" (OuterVolumeSpecName: "kube-api-access-4qf59") pod "89df2eb9-03ae-46ec-b945-cc0ae5d35c17" (UID: "89df2eb9-03ae-46ec-b945-cc0ae5d35c17"). InnerVolumeSpecName "kube-api-access-4qf59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:43:19.853010 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.852980 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgd4h\" (UniqueName: \"kubernetes.io/projected/28e98d38-5855-4431-a5b1-d2772e495577-kube-api-access-sgd4h\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:43:19.853010 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:19.853005 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qf59\" (UniqueName: \"kubernetes.io/projected/89df2eb9-03ae-46ec-b945-cc0ae5d35c17-kube-api-access-4qf59\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:43:20.384019 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.383986 2531 generic.go:356] "Generic (PLEG): container finished" podID="89df2eb9-03ae-46ec-b945-cc0ae5d35c17" containerID="38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec" exitCode=0 Jun 10 17:43:20.384424 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.384026 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" Jun 10 17:43:20.384424 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.384062 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" event={"ID":"89df2eb9-03ae-46ec-b945-cc0ae5d35c17","Type":"ContainerDied","Data":"38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec"} Jun 10 17:43:20.384424 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.384097 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xkjgm" event={"ID":"89df2eb9-03ae-46ec-b945-cc0ae5d35c17","Type":"ContainerDied","Data":"aa32cafc195f0f29478eb218d048e1e8563f856700228b704308e308d16d8486"} Jun 10 17:43:20.384424 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.384115 2531 scope.go:117] "RemoveContainer" containerID="38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec" Jun 10 17:43:20.385517 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.385356 2531 generic.go:356] "Generic (PLEG): container finished" podID="28e98d38-5855-4431-a5b1-d2772e495577" containerID="fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71" exitCode=0 Jun 10 17:43:20.385517 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.385406 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8594bb864b-qcwt9" Jun 10 17:43:20.385517 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.385433 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8594bb864b-qcwt9" event={"ID":"28e98d38-5855-4431-a5b1-d2772e495577","Type":"ContainerDied","Data":"fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71"} Jun 10 17:43:20.385517 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.385471 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8594bb864b-qcwt9" event={"ID":"28e98d38-5855-4431-a5b1-d2772e495577","Type":"ContainerDied","Data":"200bfb7f077e8540781e5711ce4065dab2c87ec367e09ef0d610d50ae3eca945"} Jun 10 17:43:20.387319 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.387298 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" event={"ID":"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9","Type":"ContainerStarted","Data":"34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac"} Jun 10 17:43:20.393508 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.393489 2531 scope.go:117] "RemoveContainer" containerID="38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec" Jun 10 17:43:20.393787 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:43:20.393767 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec\": container with ID starting with 38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec not found: ID does not exist" containerID="38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec" Jun 10 17:43:20.393883 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.393800 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec"} err="failed to get container status \"38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec\": rpc error: code = NotFound desc = could not find container \"38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec\": container with ID starting with 38ce2760ceb291013975a17f6e853c0d474336dbae7b0b2e4702007a50c70dec not found: ID does not exist" Jun 10 17:43:20.393883 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.393823 2531 scope.go:117] "RemoveContainer" containerID="fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71" Jun 10 17:43:20.401139 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.401124 2531 scope.go:117] "RemoveContainer" containerID="fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71" Jun 10 17:43:20.401368 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:43:20.401350 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71\": container with ID starting with fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71 not found: ID does not exist" containerID="fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71" Jun 10 17:43:20.401418 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.401374 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71"} err="failed to get container status \"fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71\": rpc error: code = NotFound desc = could not find container \"fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71\": container with ID starting with fe567af4f043a64a50cd0e8437f3c3c18a39e8ef7007c2dfb5d34bd95a024c71 not found: ID does not exist" Jun 10 17:43:20.403674 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.403611 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" podStartSLOduration=2.064473369 podStartE2EDuration="2.403598404s" totalImagesPullingTime="339.125035ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:43:20.402236911 +0000 UTC m=+710.370442949" watchObservedRunningTime="2026-06-10 17:43:20.403598404 +0000 UTC m=+710.371804444" Jun 10 17:43:20.419689 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.419663 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xkjgm"] Jun 10 17:43:20.428607 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.428586 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xkjgm"] Jun 10 17:43:20.441016 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.440985 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8594bb864b-qcwt9"] Jun 10 17:43:20.442364 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.442340 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8594bb864b-qcwt9"] Jun 10 17:43:20.469143 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.469120 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e98d38-5855-4431-a5b1-d2772e495577" path="/var/lib/kubelet/pods/28e98d38-5855-4431-a5b1-d2772e495577/volumes" Jun 10 17:43:20.469432 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:20.469420 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89df2eb9-03ae-46ec-b945-cc0ae5d35c17" path="/var/lib/kubelet/pods/89df2eb9-03ae-46ec-b945-cc0ae5d35c17/volumes" Jun 10 17:43:23.969051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.969019 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d7b7879f8-qs48j"] Jun 10 17:43:23.969413 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.969401 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89df2eb9-03ae-46ec-b945-cc0ae5d35c17" containerName="authorino" Jun 10 17:43:23.969454 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.969419 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="89df2eb9-03ae-46ec-b945-cc0ae5d35c17" containerName="authorino" Jun 10 17:43:23.969454 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.969434 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e98d38-5855-4431-a5b1-d2772e495577" containerName="authorino" Jun 10 17:43:23.969454 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.969440 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e98d38-5855-4431-a5b1-d2772e495577" containerName="authorino" Jun 10 17:43:23.969544 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.969503 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="89df2eb9-03ae-46ec-b945-cc0ae5d35c17" containerName="authorino" Jun 10 17:43:23.969544 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.969512 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e98d38-5855-4431-a5b1-d2772e495577" containerName="authorino" Jun 10 17:43:23.973794 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.973776 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:23.975767 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.975725 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 10 17:43:23.975876 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.975829 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 10 17:43:23.975876 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.975862 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-s6fww\"" Jun 10 17:43:23.981289 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.981268 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Jun 10 17:43:23.982586 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:23.982567 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d7b7879f8-qs48j"] Jun 10 17:43:24.090593 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.090553 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f498e1c-fec4-4cf4-ae08-0794568b7d58-odh-trusted-ca-bundle\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.090782 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.090611 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc45j\" (UniqueName: \"kubernetes.io/projected/9f498e1c-fec4-4cf4-ae08-0794568b7d58-kube-api-access-xc45j\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.090782 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.090702 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.131576 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.131544 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-bdd4f677f-r26bp"] Jun 10 17:43:24.134687 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.134670 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.144641 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.144619 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bdd4f677f-r26bp"] Jun 10 17:43:24.191219 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.191195 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc45j\" (UniqueName: \"kubernetes.io/projected/9f498e1c-fec4-4cf4-ae08-0794568b7d58-kube-api-access-xc45j\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.191320 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.191250 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.191320 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.191304 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f498e1c-fec4-4cf4-ae08-0794568b7d58-odh-trusted-ca-bundle\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.191402 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:43:24.191383 2531 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 10 17:43:24.191465 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:43:24.191451 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert podName:9f498e1c-fec4-4cf4-ae08-0794568b7d58 nodeName:}" failed. No retries permitted until 2026-06-10 17:43:24.691435683 +0000 UTC m=+714.659641701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert") pod "maas-controller-d7b7879f8-qs48j" (UID: "9f498e1c-fec4-4cf4-ae08-0794568b7d58") : secret "maas-controller-webhook-cert" not found Jun 10 17:43:24.192088 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.192072 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f498e1c-fec4-4cf4-ae08-0794568b7d58-odh-trusted-ca-bundle\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.199165 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.199143 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc45j\" (UniqueName: \"kubernetes.io/projected/9f498e1c-fec4-4cf4-ae08-0794568b7d58-kube-api-access-xc45j\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.291913 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.291882 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-odh-trusted-ca-bundle\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.292062 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.291960 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkf88\" (UniqueName: \"kubernetes.io/projected/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-kube-api-access-nkf88\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.292062 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.292013 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-webhook-cert\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.392673 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.392639 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-odh-trusted-ca-bundle\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.392812 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.392699 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkf88\" (UniqueName: \"kubernetes.io/projected/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-kube-api-access-nkf88\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.392855 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.392801 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-webhook-cert\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.393502 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.393477 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-odh-trusted-ca-bundle\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.395230 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.395215 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-webhook-cert\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.400402 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.400380 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkf88\" (UniqueName: \"kubernetes.io/projected/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-kube-api-access-nkf88\") pod \"maas-controller-bdd4f677f-r26bp\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.444783 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.444752 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:24.574091 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.574068 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bdd4f677f-r26bp"] Jun 10 17:43:24.576068 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:43:24.576041 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ebbadc_bab4_4d71_b51f_c39eb70fa4d9.slice/crio-5713ede8055a040ed8a1544a4fd6bbfbe82007a218c1bafa91f65a863619822d WatchSource:0}: Error finding container 5713ede8055a040ed8a1544a4fd6bbfbe82007a218c1bafa91f65a863619822d: Status 404 returned error can't find the container with id 5713ede8055a040ed8a1544a4fd6bbfbe82007a218c1bafa91f65a863619822d Jun 10 17:43:24.577391 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.577375 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 17:43:24.696079 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.696053 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.698578 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.698553 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert\") pod \"maas-controller-d7b7879f8-qs48j\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:24.884703 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:24.884624 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:25.012097 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:25.012072 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d7b7879f8-qs48j"] Jun 10 17:43:25.014119 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:43:25.014094 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f498e1c_fec4_4cf4_ae08_0794568b7d58.slice/crio-2954df4ce2da94f96b27292020698db55f8c1dbef7d5b1ab9387585bdc693f38 WatchSource:0}: Error finding container 2954df4ce2da94f96b27292020698db55f8c1dbef7d5b1ab9387585bdc693f38: Status 404 returned error can't find the container with id 2954df4ce2da94f96b27292020698db55f8c1dbef7d5b1ab9387585bdc693f38 Jun 10 17:43:25.412669 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:25.412617 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bdd4f677f-r26bp" event={"ID":"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9","Type":"ContainerStarted","Data":"5713ede8055a040ed8a1544a4fd6bbfbe82007a218c1bafa91f65a863619822d"} Jun 10 17:43:25.414090 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:25.414060 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d7b7879f8-qs48j" event={"ID":"9f498e1c-fec4-4cf4-ae08-0794568b7d58","Type":"ContainerStarted","Data":"2954df4ce2da94f96b27292020698db55f8c1dbef7d5b1ab9387585bdc693f38"} Jun 10 17:43:27.427838 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:27.427798 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bdd4f677f-r26bp" event={"ID":"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9","Type":"ContainerStarted","Data":"d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119"} Jun 10 17:43:27.428303 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:27.427873 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:27.429373 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:27.429347 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d7b7879f8-qs48j" event={"ID":"9f498e1c-fec4-4cf4-ae08-0794568b7d58","Type":"ContainerStarted","Data":"784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a"} Jun 10 17:43:27.429495 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:27.429478 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:27.443996 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:27.443950 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-bdd4f677f-r26bp" podStartSLOduration=0.751412623 podStartE2EDuration="3.443918224s" totalImagesPullingTime="2.692505601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:43:27.442227759 +0000 UTC m=+717.410433798" watchObservedRunningTime="2026-06-10 17:43:27.443918224 +0000 UTC m=+717.412124263" Jun 10 17:43:27.457463 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:27.457419 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d7b7879f8-qs48j" podStartSLOduration=2.193455604 podStartE2EDuration="4.45740681s" totalImagesPullingTime="2.263951206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:43:27.456384335 +0000 UTC m=+717.424590375" watchObservedRunningTime="2026-06-10 17:43:27.45740681 +0000 UTC m=+717.425612848" Jun 10 17:43:38.439750 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.439718 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:43:38.440207 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.439792 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:38.492536 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.492504 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d7b7879f8-qs48j"] Jun 10 17:43:38.492682 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.492665 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d7b7879f8-qs48j" podUID="9f498e1c-fec4-4cf4-ae08-0794568b7d58" containerName="manager" containerID="cri-o://784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a" gracePeriod=30 Jun 10 17:43:38.742723 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.742696 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:38.825162 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.825120 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f498e1c-fec4-4cf4-ae08-0794568b7d58-odh-trusted-ca-bundle\") pod \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " Jun 10 17:43:38.825336 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.825180 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xc45j\" (UniqueName: \"kubernetes.io/projected/9f498e1c-fec4-4cf4-ae08-0794568b7d58-kube-api-access-xc45j\") pod \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " Jun 10 17:43:38.825336 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.825204 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert\") pod \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\" (UID: \"9f498e1c-fec4-4cf4-ae08-0794568b7d58\") " Jun 10 17:43:38.825474 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.825449 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f498e1c-fec4-4cf4-ae08-0794568b7d58-odh-trusted-ca-bundle" (OuterVolumeSpecName: "odh-trusted-ca-bundle") pod "9f498e1c-fec4-4cf4-ae08-0794568b7d58" (UID: "9f498e1c-fec4-4cf4-ae08-0794568b7d58"). InnerVolumeSpecName "odh-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:43:38.827480 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.827451 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9f498e1c-fec4-4cf4-ae08-0794568b7d58" (UID: "9f498e1c-fec4-4cf4-ae08-0794568b7d58"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:43:38.827601 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.827525 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f498e1c-fec4-4cf4-ae08-0794568b7d58-kube-api-access-xc45j" (OuterVolumeSpecName: "kube-api-access-xc45j") pod "9f498e1c-fec4-4cf4-ae08-0794568b7d58" (UID: "9f498e1c-fec4-4cf4-ae08-0794568b7d58"). InnerVolumeSpecName "kube-api-access-xc45j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:43:38.926630 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.926597 2531 reconciler_common.go:299] "Volume detached for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f498e1c-fec4-4cf4-ae08-0794568b7d58-odh-trusted-ca-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:43:38.926630 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.926625 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc45j\" (UniqueName: \"kubernetes.io/projected/9f498e1c-fec4-4cf4-ae08-0794568b7d58-kube-api-access-xc45j\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:43:38.926630 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:38.926635 2531 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f498e1c-fec4-4cf4-ae08-0794568b7d58-webhook-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:43:39.482860 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.482818 2531 generic.go:356] "Generic (PLEG): container finished" podID="9f498e1c-fec4-4cf4-ae08-0794568b7d58" containerID="784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a" exitCode=0 Jun 10 17:43:39.483266 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.482871 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d7b7879f8-qs48j" Jun 10 17:43:39.483266 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.482899 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d7b7879f8-qs48j" event={"ID":"9f498e1c-fec4-4cf4-ae08-0794568b7d58","Type":"ContainerDied","Data":"784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a"} Jun 10 17:43:39.483266 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.482965 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d7b7879f8-qs48j" event={"ID":"9f498e1c-fec4-4cf4-ae08-0794568b7d58","Type":"ContainerDied","Data":"2954df4ce2da94f96b27292020698db55f8c1dbef7d5b1ab9387585bdc693f38"} Jun 10 17:43:39.483266 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.482990 2531 scope.go:117] "RemoveContainer" containerID="784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a" Jun 10 17:43:39.491844 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.491828 2531 scope.go:117] "RemoveContainer" containerID="784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a" Jun 10 17:43:39.492142 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:43:39.492120 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a\": container with ID starting with 784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a not found: ID does not exist" containerID="784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a" Jun 10 17:43:39.492210 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.492151 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a"} err="failed to get container status \"784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a\": rpc error: code = NotFound desc = could not find container \"784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a\": container with ID starting with 784a25c193f36310b763ffb7b072eedac290912cc5779b64f1b9a415c310889a not found: ID does not exist" Jun 10 17:43:39.502415 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.502396 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d7b7879f8-qs48j"] Jun 10 17:43:39.505652 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:39.505632 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d7b7879f8-qs48j"] Jun 10 17:43:40.469620 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:40.469584 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f498e1c-fec4-4cf4-ae08-0794568b7d58" path="/var/lib/kubelet/pods/9f498e1c-fec4-4cf4-ae08-0794568b7d58/volumes" Jun 10 17:43:57.772510 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.772474 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-784fdbf859-82rcr"] Jun 10 17:43:57.772984 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.772795 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f498e1c-fec4-4cf4-ae08-0794568b7d58" containerName="manager" Jun 10 17:43:57.772984 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.772807 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f498e1c-fec4-4cf4-ae08-0794568b7d58" containerName="manager" Jun 10 17:43:57.772984 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.772874 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f498e1c-fec4-4cf4-ae08-0794568b7d58" containerName="manager" Jun 10 17:43:57.779243 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.779221 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:43:57.781260 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.781244 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 10 17:43:57.781426 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.781406 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-tmk6b\"" Jun 10 17:43:57.790631 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.790611 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-784fdbf859-82rcr"] Jun 10 17:43:57.886676 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.886639 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fac72386-d677-4971-a124-be3582bde7c0-maas-api-tls\") pod \"maas-api-784fdbf859-82rcr\" (UID: \"fac72386-d677-4971-a124-be3582bde7c0\") " pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:43:57.886676 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.886676 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77pl5\" (UniqueName: \"kubernetes.io/projected/fac72386-d677-4971-a124-be3582bde7c0-kube-api-access-77pl5\") pod \"maas-api-784fdbf859-82rcr\" (UID: \"fac72386-d677-4971-a124-be3582bde7c0\") " pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:43:57.987727 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.987693 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fac72386-d677-4971-a124-be3582bde7c0-maas-api-tls\") pod \"maas-api-784fdbf859-82rcr\" (UID: \"fac72386-d677-4971-a124-be3582bde7c0\") " pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:43:57.987903 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.987731 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77pl5\" (UniqueName: \"kubernetes.io/projected/fac72386-d677-4971-a124-be3582bde7c0-kube-api-access-77pl5\") pod \"maas-api-784fdbf859-82rcr\" (UID: \"fac72386-d677-4971-a124-be3582bde7c0\") " pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:43:57.990364 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.990337 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fac72386-d677-4971-a124-be3582bde7c0-maas-api-tls\") pod \"maas-api-784fdbf859-82rcr\" (UID: \"fac72386-d677-4971-a124-be3582bde7c0\") " pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:43:57.995674 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:57.995651 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77pl5\" (UniqueName: \"kubernetes.io/projected/fac72386-d677-4971-a124-be3582bde7c0-kube-api-access-77pl5\") pod \"maas-api-784fdbf859-82rcr\" (UID: \"fac72386-d677-4971-a124-be3582bde7c0\") " pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:43:58.089283 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.089202 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:43:58.224610 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.224582 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-784fdbf859-82rcr"] Jun 10 17:43:58.229187 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:43:58.229124 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac72386_d677_4971_a124_be3582bde7c0.slice/crio-48b31e7180d423a1404287acc65637350900bb27fde969ebad1a3ed1d04b9342 WatchSource:0}: Error finding container 48b31e7180d423a1404287acc65637350900bb27fde969ebad1a3ed1d04b9342: Status 404 returned error can't find the container with id 48b31e7180d423a1404287acc65637350900bb27fde969ebad1a3ed1d04b9342 Jun 10 17:43:58.268634 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.268606 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-8wd86"] Jun 10 17:43:58.274086 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.274064 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" Jun 10 17:43:58.277059 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.277036 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 10 17:43:58.277159 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.277036 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bv9rd\"" Jun 10 17:43:58.283577 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.283558 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-8wd86"] Jun 10 17:43:58.392655 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.392577 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vprfg\" (UniqueName: \"kubernetes.io/projected/f75622db-3970-4410-8962-fe8f3bde954a-kube-api-access-vprfg\") pod \"payload-processing-699b6cd6c-8wd86\" (UID: \"f75622db-3970-4410-8962-fe8f3bde954a\") " pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" Jun 10 17:43:58.494262 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.494226 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprfg\" (UniqueName: \"kubernetes.io/projected/f75622db-3970-4410-8962-fe8f3bde954a-kube-api-access-vprfg\") pod \"payload-processing-699b6cd6c-8wd86\" (UID: \"f75622db-3970-4410-8962-fe8f3bde954a\") " pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" Jun 10 17:43:58.502014 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.501991 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vprfg\" (UniqueName: \"kubernetes.io/projected/f75622db-3970-4410-8962-fe8f3bde954a-kube-api-access-vprfg\") pod \"payload-processing-699b6cd6c-8wd86\" (UID: \"f75622db-3970-4410-8962-fe8f3bde954a\") " pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" Jun 10 17:43:58.564815 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.564773 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-784fdbf859-82rcr" event={"ID":"fac72386-d677-4971-a124-be3582bde7c0","Type":"ContainerStarted","Data":"48b31e7180d423a1404287acc65637350900bb27fde969ebad1a3ed1d04b9342"} Jun 10 17:43:58.584616 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.584585 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" Jun 10 17:43:58.715492 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:58.715466 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-8wd86"] Jun 10 17:43:58.717603 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:43:58.717571 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf75622db_3970_4410_8962_fe8f3bde954a.slice/crio-3e146fe2bb6062e3257778cd914f8440a5c18ac43b07767e4bdf23b9ab53f585 WatchSource:0}: Error finding container 3e146fe2bb6062e3257778cd914f8440a5c18ac43b07767e4bdf23b9ab53f585: Status 404 returned error can't find the container with id 3e146fe2bb6062e3257778cd914f8440a5c18ac43b07767e4bdf23b9ab53f585 Jun 10 17:43:59.575404 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:43:59.575326 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" event={"ID":"f75622db-3970-4410-8962-fe8f3bde954a","Type":"ContainerStarted","Data":"3e146fe2bb6062e3257778cd914f8440a5c18ac43b07767e4bdf23b9ab53f585"} Jun 10 17:44:02.591759 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:02.591719 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-784fdbf859-82rcr" event={"ID":"fac72386-d677-4971-a124-be3582bde7c0","Type":"ContainerStarted","Data":"15e68184bf74b25c09a0bb1254a2d9eb4fc8daeb66f813289664a8dce87af356"} Jun 10 17:44:02.592194 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:02.591792 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:44:02.593256 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:02.593232 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" event={"ID":"f75622db-3970-4410-8962-fe8f3bde954a","Type":"ContainerStarted","Data":"fa5bcbdea07795bf136866c04cada29bb52a61838841b53a080442283bd82e21"} Jun 10 17:44:02.593385 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:02.593366 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" Jun 10 17:44:02.609818 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:02.609778 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-784fdbf859-82rcr" podStartSLOduration=2.2340702 podStartE2EDuration="5.609766185s" totalImagesPullingTime="3.375695985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:43:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:44:02.608798688 +0000 UTC m=+752.577004732" watchObservedRunningTime="2026-06-10 17:44:02.609766185 +0000 UTC m=+752.577972226" Jun 10 17:44:02.623318 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:02.623281 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" podStartSLOduration=1.663980648 podStartE2EDuration="4.623269139s" totalImagesPullingTime="2.959288491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:44:02.621973861 +0000 UTC m=+752.590179902" watchObservedRunningTime="2026-06-10 17:44:02.623269139 +0000 UTC m=+752.591475178" Jun 10 17:44:05.870880 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:05.870823 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-98d67 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": read tcp 10.134.0.2:60716->10.134.0.30:15021: read: connection reset by peer" start-of-body= Jun 10 17:44:05.871357 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:05.870906 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" podUID="a533ebf2-5111-41c2-8322-c7a463e38275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": read tcp 10.134.0.2:60716->10.134.0.30:15021: read: connection reset by peer" Jun 10 17:44:06.613427 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:06.613388 2531 generic.go:356] "Generic (PLEG): container finished" podID="a533ebf2-5111-41c2-8322-c7a463e38275" containerID="627f7cdf112d08681a0b26ac59f6fdcbb14d76b8b512f6279624ca66418ce09f" exitCode=0 Jun 10 17:44:06.613599 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:06.613460 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" event={"ID":"a533ebf2-5111-41c2-8322-c7a463e38275","Type":"ContainerDied","Data":"627f7cdf112d08681a0b26ac59f6fdcbb14d76b8b512f6279624ca66418ce09f"} Jun 10 17:44:06.613775 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:06.613760 2531 scope.go:117] "RemoveContainer" containerID="627f7cdf112d08681a0b26ac59f6fdcbb14d76b8b512f6279624ca66418ce09f" Jun 10 17:44:06.614633 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:06.614607 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:44:06.614717 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:06.614670 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:44:06.614717 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:06.614701 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 10 17:44:07.619454 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:07.619420 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" event={"ID":"a533ebf2-5111-41c2-8322-c7a463e38275","Type":"ContainerStarted","Data":"a741e052870633fa8df244edf97fe5df1432b1533398c2ec07280b78a8164f69"} Jun 10 17:44:08.254288 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:08.254244 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:44:08.254467 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:08.254442 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-98d67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jun 10 17:44:08.254512 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:08.254494 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" podUID="a533ebf2-5111-41c2-8322-c7a463e38275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jun 10 17:44:08.604379 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:08.604308 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-784fdbf859-82rcr" Jun 10 17:44:09.253873 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:09.253820 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-98d67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jun 10 17:44:09.254311 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:09.253898 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" podUID="a533ebf2-5111-41c2-8322-c7a463e38275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jun 10 17:44:10.253915 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:10.253869 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-98d67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jun 10 17:44:10.254326 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:10.253991 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" podUID="a533ebf2-5111-41c2-8322-c7a463e38275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jun 10 17:44:10.359992 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:10.359959 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:44:10.360554 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:10.360499 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" containerID="cri-o://8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c" gracePeriod=30 Jun 10 17:44:11.254656 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:11.254608 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-98d67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jun 10 17:44:11.255089 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:11.254683 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" podUID="a533ebf2-5111-41c2-8322-c7a463e38275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jun 10 17:44:12.010883 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.010861 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:12.123209 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.123129 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9ksx\" (UniqueName: \"kubernetes.io/projected/7dbb1199-37de-4be0-8687-09c5d6e7dfa6-kube-api-access-w9ksx\") pod \"7dbb1199-37de-4be0-8687-09c5d6e7dfa6\" (UID: \"7dbb1199-37de-4be0-8687-09c5d6e7dfa6\") " Jun 10 17:44:12.125322 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.125288 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dbb1199-37de-4be0-8687-09c5d6e7dfa6-kube-api-access-w9ksx" (OuterVolumeSpecName: "kube-api-access-w9ksx") pod "7dbb1199-37de-4be0-8687-09c5d6e7dfa6" (UID: "7dbb1199-37de-4be0-8687-09c5d6e7dfa6"). InnerVolumeSpecName "kube-api-access-w9ksx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:44:12.224349 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.224320 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9ksx\" (UniqueName: \"kubernetes.io/projected/7dbb1199-37de-4be0-8687-09c5d6e7dfa6-kube-api-access-w9ksx\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:44:12.257624 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.257596 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:44:12.258070 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.257863 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:44:12.258696 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.258680 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-98d67" Jun 10 17:44:12.644043 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.644007 2531 generic.go:356] "Generic (PLEG): container finished" podID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerID="8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c" exitCode=143 Jun 10 17:44:12.644217 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.644054 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:12.644217 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.644094 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7dbb1199-37de-4be0-8687-09c5d6e7dfa6","Type":"ContainerDied","Data":"8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c"} Jun 10 17:44:12.644217 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.644144 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7dbb1199-37de-4be0-8687-09c5d6e7dfa6","Type":"ContainerDied","Data":"d4bff0d2ae30025176009e5941602eebb37053649897db38d74b2a2115a77e64"} Jun 10 17:44:12.644217 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.644163 2531 scope.go:117] "RemoveContainer" containerID="8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c" Jun 10 17:44:12.653227 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.653212 2531 scope.go:117] "RemoveContainer" containerID="8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c" Jun 10 17:44:12.653462 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:44:12.653445 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c\": container with ID starting with 8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c not found: ID does not exist" containerID="8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c" Jun 10 17:44:12.653525 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.653467 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c"} err="failed to get container status \"8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c\": rpc error: code = NotFound desc = could not find container \"8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c\": container with ID starting with 8c656fa0a841de2346e2905acf92d3039ce8de7fad0be8ee13c57b52b7e8a36c not found: ID does not exist" Jun 10 17:44:12.660699 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.660681 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:44:12.664526 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.664507 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:44:12.686503 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.686479 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:44:12.686793 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.686781 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" Jun 10 17:44:12.686836 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.686795 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" Jun 10 17:44:12.686869 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.686847 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" containerName="keycloak" Jun 10 17:44:12.689371 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.689350 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:12.691631 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.691611 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 10 17:44:12.691712 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.691682 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 10 17:44:12.691712 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.691688 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-6zd5n\"" Jun 10 17:44:12.691712 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.691699 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 10 17:44:12.691845 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.691689 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 10 17:44:12.698598 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.698575 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:44:12.829290 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.829250 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/8d2969b1-feb7-4b90-a9d3-7eb81bc66c29-test-realms\") pod \"maas-keycloak-0\" (UID: \"8d2969b1-feb7-4b90-a9d3-7eb81bc66c29\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:12.829463 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.829440 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bl6f\" (UniqueName: \"kubernetes.io/projected/8d2969b1-feb7-4b90-a9d3-7eb81bc66c29-kube-api-access-6bl6f\") pod \"maas-keycloak-0\" (UID: \"8d2969b1-feb7-4b90-a9d3-7eb81bc66c29\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:12.929946 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.929917 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bl6f\" (UniqueName: \"kubernetes.io/projected/8d2969b1-feb7-4b90-a9d3-7eb81bc66c29-kube-api-access-6bl6f\") pod \"maas-keycloak-0\" (UID: \"8d2969b1-feb7-4b90-a9d3-7eb81bc66c29\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:12.930078 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.929979 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/8d2969b1-feb7-4b90-a9d3-7eb81bc66c29-test-realms\") pod \"maas-keycloak-0\" (UID: \"8d2969b1-feb7-4b90-a9d3-7eb81bc66c29\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:12.930595 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.930575 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/8d2969b1-feb7-4b90-a9d3-7eb81bc66c29-test-realms\") pod \"maas-keycloak-0\" (UID: \"8d2969b1-feb7-4b90-a9d3-7eb81bc66c29\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:12.938375 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:12.938345 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bl6f\" (UniqueName: \"kubernetes.io/projected/8d2969b1-feb7-4b90-a9d3-7eb81bc66c29-kube-api-access-6bl6f\") pod \"maas-keycloak-0\" (UID: \"8d2969b1-feb7-4b90-a9d3-7eb81bc66c29\") " pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:13.000144 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:13.000117 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:13.131291 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:13.131180 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 10 17:44:13.133446 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:44:13.133417 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2969b1_feb7_4b90_a9d3_7eb81bc66c29.slice/crio-3c2dc52d68a1660bb8d7b1bf1b58b278947db17398b38bb747f01f3587bbe190 WatchSource:0}: Error finding container 3c2dc52d68a1660bb8d7b1bf1b58b278947db17398b38bb747f01f3587bbe190: Status 404 returned error can't find the container with id 3c2dc52d68a1660bb8d7b1bf1b58b278947db17398b38bb747f01f3587bbe190 Jun 10 17:44:13.601074 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:13.601046 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-8wd86" Jun 10 17:44:13.659677 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:13.659649 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"8d2969b1-feb7-4b90-a9d3-7eb81bc66c29","Type":"ContainerStarted","Data":"3c2dc52d68a1660bb8d7b1bf1b58b278947db17398b38bb747f01f3587bbe190"} Jun 10 17:44:14.472777 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:14.472729 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dbb1199-37de-4be0-8687-09c5d6e7dfa6" path="/var/lib/kubelet/pods/7dbb1199-37de-4be0-8687-09c5d6e7dfa6/volumes" Jun 10 17:44:14.669322 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:14.669280 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"8d2969b1-feb7-4b90-a9d3-7eb81bc66c29","Type":"ContainerStarted","Data":"87f9b09433e247d734be05728c1e55cb188e9574ab5e367aa7049fc71a0f723a"} Jun 10 17:44:14.687717 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:14.687659 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.189062725 podStartE2EDuration="2.687638076s" totalImagesPullingTime="498.575351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:44:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:44:14.68583616 +0000 UTC m=+764.654042200" watchObservedRunningTime="2026-06-10 17:44:14.687638076 +0000 UTC m=+764.655844116" Jun 10 17:44:15.000976 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:15.000915 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:15.002815 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:15.002780 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:16.001511 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:16.001461 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:17.001609 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:17.001559 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:18.001254 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:18.001205 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:19.001474 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:19.001428 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:20.000670 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:20.000617 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:21.001444 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:21.001381 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:22.001365 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:22.001306 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:23.001196 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:23.001150 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:23.001718 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:23.001540 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:24.000569 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:24.000515 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:25.001222 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:25.001171 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:26.000715 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:26.000663 2531 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jun 10 17:44:27.086331 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:27.086285 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:27.099723 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:27.099654 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="8d2969b1-feb7-4b90-a9d3-7eb81bc66c29" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 10 17:44:37.094008 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:37.093973 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 10 17:44:49.550807 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.550730 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74cf5dc58b-gtpgn"] Jun 10 17:44:49.551251 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.550967 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" podUID="f1312fcf-04b2-4fe2-a941-2e2a9a4794e9" containerName="authorino" containerID="cri-o://34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac" gracePeriod=30 Jun 10 17:44:49.804215 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.804159 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:44:49.860727 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.860687 2531 generic.go:356] "Generic (PLEG): container finished" podID="f1312fcf-04b2-4fe2-a941-2e2a9a4794e9" containerID="34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac" exitCode=0 Jun 10 17:44:49.860873 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.860732 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" Jun 10 17:44:49.860873 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.860735 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" event={"ID":"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9","Type":"ContainerDied","Data":"34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac"} Jun 10 17:44:49.860873 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.860864 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74cf5dc58b-gtpgn" event={"ID":"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9","Type":"ContainerDied","Data":"6ef015cbf669fa2c8e8b71a34a0674e1b6d799eb5b3528c657433af516fedb58"} Jun 10 17:44:49.861005 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.860892 2531 scope.go:117] "RemoveContainer" containerID="34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac" Jun 10 17:44:49.869011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.868995 2531 scope.go:117] "RemoveContainer" containerID="34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac" Jun 10 17:44:49.869274 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:44:49.869257 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac\": container with ID starting with 34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac not found: ID does not exist" containerID="34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac" Jun 10 17:44:49.869329 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.869283 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac"} err="failed to get container status \"34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac\": rpc error: code = NotFound desc = could not find container \"34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac\": container with ID starting with 34e793815167a1d18353012ff9f56d9bf685bcf86a5d7938added68264437eac not found: ID does not exist" Jun 10 17:44:49.976584 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.976551 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lwr8\" (UniqueName: \"kubernetes.io/projected/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-kube-api-access-7lwr8\") pod \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\" (UID: \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\") " Jun 10 17:44:49.976740 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.976621 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-tls-cert\") pod \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\" (UID: \"f1312fcf-04b2-4fe2-a941-2e2a9a4794e9\") " Jun 10 17:44:49.979353 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.979318 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-kube-api-access-7lwr8" (OuterVolumeSpecName: "kube-api-access-7lwr8") pod "f1312fcf-04b2-4fe2-a941-2e2a9a4794e9" (UID: "f1312fcf-04b2-4fe2-a941-2e2a9a4794e9"). InnerVolumeSpecName "kube-api-access-7lwr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:44:49.988863 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:49.988836 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f1312fcf-04b2-4fe2-a941-2e2a9a4794e9" (UID: "f1312fcf-04b2-4fe2-a941-2e2a9a4794e9"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:44:50.077613 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:50.077531 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lwr8\" (UniqueName: \"kubernetes.io/projected/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-kube-api-access-7lwr8\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:44:50.077613 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:50.077564 2531 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9-tls-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:44:50.188433 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:50.188402 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74cf5dc58b-gtpgn"] Jun 10 17:44:50.196973 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:50.196931 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-74cf5dc58b-gtpgn"] Jun 10 17:44:50.470175 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:44:50.470092 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1312fcf-04b2-4fe2-a941-2e2a9a4794e9" path="/var/lib/kubelet/pods/f1312fcf-04b2-4fe2-a941-2e2a9a4794e9/volumes" Jun 10 17:45:01.596338 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.596295 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks"] Jun 10 17:45:01.596866 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.596845 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1312fcf-04b2-4fe2-a941-2e2a9a4794e9" containerName="authorino" Jun 10 17:45:01.597026 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.596869 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1312fcf-04b2-4fe2-a941-2e2a9a4794e9" containerName="authorino" Jun 10 17:45:01.597026 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.596978 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1312fcf-04b2-4fe2-a941-2e2a9a4794e9" containerName="authorino" Jun 10 17:45:01.600206 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.600183 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.603264 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.603236 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fvfqg\"" Jun 10 17:45:01.603377 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.603273 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 10 17:45:01.603377 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.603285 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 10 17:45:01.603377 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.603294 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 10 17:45:01.609585 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.609562 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks"] Jun 10 17:45:01.674548 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.674514 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.674696 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.674563 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.674696 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.674596 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.674696 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.674623 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f55e024-38a9-4a81-998f-e0d937bc6ce4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.674696 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.674639 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rkj5\" (UniqueName: \"kubernetes.io/projected/8f55e024-38a9-4a81-998f-e0d937bc6ce4-kube-api-access-4rkj5\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.674696 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.674665 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.775574 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.775529 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.775754 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.775684 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.775824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.775753 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.775824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.775790 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.775963 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.775920 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f55e024-38a9-4a81-998f-e0d937bc6ce4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.776011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.775980 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkj5\" (UniqueName: \"kubernetes.io/projected/8f55e024-38a9-4a81-998f-e0d937bc6ce4-kube-api-access-4rkj5\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.776011 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.775991 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.776189 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.776168 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.776280 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.776200 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.777943 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.777921 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f55e024-38a9-4a81-998f-e0d937bc6ce4-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.778413 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.778396 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f55e024-38a9-4a81-998f-e0d937bc6ce4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.783375 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.783355 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rkj5\" (UniqueName: \"kubernetes.io/projected/8f55e024-38a9-4a81-998f-e0d937bc6ce4-kube-api-access-4rkj5\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks\" (UID: \"8f55e024-38a9-4a81-998f-e0d937bc6ce4\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:01.911040 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:01.910903 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:45:02.045652 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:02.045627 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks"] Jun 10 17:45:02.047808 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:45:02.047779 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f55e024_38a9_4a81_998f_e0d937bc6ce4.slice/crio-da20898bb6e8e1c98e100c43581ba438c7f62b6cafb655818a851b08b83eeb6e WatchSource:0}: Error finding container da20898bb6e8e1c98e100c43581ba438c7f62b6cafb655818a851b08b83eeb6e: Status 404 returned error can't find the container with id da20898bb6e8e1c98e100c43581ba438c7f62b6cafb655818a851b08b83eeb6e Jun 10 17:45:02.920306 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:02.920271 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" event={"ID":"8f55e024-38a9-4a81-998f-e0d937bc6ce4","Type":"ContainerStarted","Data":"da20898bb6e8e1c98e100c43581ba438c7f62b6cafb655818a851b08b83eeb6e"} Jun 10 17:45:04.181442 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.181405 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr"] Jun 10 17:45:04.186745 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.186719 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.188811 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.188789 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 10 17:45:04.193868 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.193843 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr"] Jun 10 17:45:04.200786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.200691 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dchz7\" (UniqueName: \"kubernetes.io/projected/6a6e14bb-4624-41a0-bdbf-45a6805a809e-kube-api-access-dchz7\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.200786 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.200780 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.200988 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.200811 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.200988 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.200853 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a6e14bb-4624-41a0-bdbf-45a6805a809e-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.200988 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.200970 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.201150 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.201010 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.302824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.302214 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.302824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.302266 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.302824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.302320 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a6e14bb-4624-41a0-bdbf-45a6805a809e-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.302824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.302367 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.302824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.302396 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.302824 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.302427 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchz7\" (UniqueName: \"kubernetes.io/projected/6a6e14bb-4624-41a0-bdbf-45a6805a809e-kube-api-access-dchz7\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.303296 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.302888 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.303651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.303489 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.303651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.303608 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.306299 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.306279 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a6e14bb-4624-41a0-bdbf-45a6805a809e-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.306651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.306629 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a6e14bb-4624-41a0-bdbf-45a6805a809e-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.310764 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.310742 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dchz7\" (UniqueName: \"kubernetes.io/projected/6a6e14bb-4624-41a0-bdbf-45a6805a809e-kube-api-access-dchz7\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mt5jr\" (UID: \"6a6e14bb-4624-41a0-bdbf-45a6805a809e\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:04.502353 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:04.502311 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:06.356232 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:06.356202 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr"] Jun 10 17:45:06.358712 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:45:06.358689 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a6e14bb_4624_41a0_bdbf_45a6805a809e.slice/crio-2e3aaaae28103e1f566416fc909393ae01ad8dbd3b6e40ecd5ffbda4a840d646 WatchSource:0}: Error finding container 2e3aaaae28103e1f566416fc909393ae01ad8dbd3b6e40ecd5ffbda4a840d646: Status 404 returned error can't find the container with id 2e3aaaae28103e1f566416fc909393ae01ad8dbd3b6e40ecd5ffbda4a840d646 Jun 10 17:45:06.945257 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:06.945219 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" event={"ID":"8f55e024-38a9-4a81-998f-e0d937bc6ce4","Type":"ContainerStarted","Data":"148c09a3293084e826ee455beb1456fa5a1d7e4d2dd39938a402a0e21482b933"} Jun 10 17:45:06.946804 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:06.946776 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" event={"ID":"6a6e14bb-4624-41a0-bdbf-45a6805a809e","Type":"ContainerStarted","Data":"94a79734c6ff6ec7951479fe586677740b6943bed61dee1aa8b59481f84b1a80"} Jun 10 17:45:06.946965 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:06.946823 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" event={"ID":"6a6e14bb-4624-41a0-bdbf-45a6805a809e","Type":"ContainerStarted","Data":"2e3aaaae28103e1f566416fc909393ae01ad8dbd3b6e40ecd5ffbda4a840d646"} Jun 10 17:45:07.952736 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:07.952705 2531 generic.go:356] "Generic (PLEG): container finished" podID="6a6e14bb-4624-41a0-bdbf-45a6805a809e" containerID="94a79734c6ff6ec7951479fe586677740b6943bed61dee1aa8b59481f84b1a80" exitCode=0 Jun 10 17:45:07.953156 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:07.952765 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" event={"ID":"6a6e14bb-4624-41a0-bdbf-45a6805a809e","Type":"ContainerDied","Data":"94a79734c6ff6ec7951479fe586677740b6943bed61dee1aa8b59481f84b1a80"} Jun 10 17:45:09.963346 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:09.963261 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" event={"ID":"6a6e14bb-4624-41a0-bdbf-45a6805a809e","Type":"ContainerStarted","Data":"1fa607d2ca545707e31f05ad9ef08841bbcb3b4eb714771dd3ef38829b1866b9"} Jun 10 17:45:09.987665 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:09.983900 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" podStartSLOduration=3.390236172 podStartE2EDuration="5.983881551s" totalImagesPullingTime="1.315573528s" totalInitContainerRuntime="1.278071851s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:45:09.981284954 +0000 UTC m=+819.949491004" watchObservedRunningTime="2026-06-10 17:45:09.983881551 +0000 UTC m=+819.952087596" Jun 10 17:45:13.070428 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.070399 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln"] Jun 10 17:45:13.075871 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.075849 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.077912 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.077890 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 10 17:45:13.087095 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.087068 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln"] Jun 10 17:45:13.180216 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.180186 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a56e596d-753a-4cce-808d-e8a9bf864b06-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.180337 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.180227 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.180337 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.180245 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.180337 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.180291 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvrtf\" (UniqueName: \"kubernetes.io/projected/a56e596d-753a-4cce-808d-e8a9bf864b06-kube-api-access-cvrtf\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.180434 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.180382 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.180469 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.180433 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.281627 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.281590 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.281627 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.281628 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.281825 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.281648 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvrtf\" (UniqueName: \"kubernetes.io/projected/a56e596d-753a-4cce-808d-e8a9bf864b06-kube-api-access-cvrtf\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.281825 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.281695 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.281825 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.281720 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.281825 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.281752 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a56e596d-753a-4cce-808d-e8a9bf864b06-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.282100 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.282077 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.282157 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.282115 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.282157 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.282145 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.284167 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.284137 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a56e596d-753a-4cce-808d-e8a9bf864b06-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.284512 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.284493 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a56e596d-753a-4cce-808d-e8a9bf864b06-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.289083 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.289063 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvrtf\" (UniqueName: \"kubernetes.io/projected/a56e596d-753a-4cce-808d-e8a9bf864b06-kube-api-access-cvrtf\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln\" (UID: \"a56e596d-753a-4cce-808d-e8a9bf864b06\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.388962 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.388867 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:13.529001 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.528976 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln"] Jun 10 17:45:13.530640 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:45:13.530614 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56e596d_753a_4cce_808d_e8a9bf864b06.slice/crio-3787e252b04b3dbbfc68ab118a7d2bb7ffd5470c26e6db95971b43c1f559aa1e WatchSource:0}: Error finding container 3787e252b04b3dbbfc68ab118a7d2bb7ffd5470c26e6db95971b43c1f559aa1e: Status 404 returned error can't find the container with id 3787e252b04b3dbbfc68ab118a7d2bb7ffd5470c26e6db95971b43c1f559aa1e Jun 10 17:45:13.982830 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.982794 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" event={"ID":"a56e596d-753a-4cce-808d-e8a9bf864b06","Type":"ContainerStarted","Data":"f6d392692c005a8160e9aeb1b7a2976b57f674393b40c0fff0079a6890dc63d1"} Jun 10 17:45:13.983019 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:13.982864 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" event={"ID":"a56e596d-753a-4cce-808d-e8a9bf864b06","Type":"ContainerStarted","Data":"3787e252b04b3dbbfc68ab118a7d2bb7ffd5470c26e6db95971b43c1f559aa1e"} Jun 10 17:45:14.502829 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:14.502791 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:14.503325 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:14.502969 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:14.515525 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:14.515497 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:14.989240 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:14.989205 2531 generic.go:356] "Generic (PLEG): container finished" podID="a56e596d-753a-4cce-808d-e8a9bf864b06" containerID="f6d392692c005a8160e9aeb1b7a2976b57f674393b40c0fff0079a6890dc63d1" exitCode=0 Jun 10 17:45:14.989366 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:14.989283 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" event={"ID":"a56e596d-753a-4cce-808d-e8a9bf864b06","Type":"ContainerDied","Data":"f6d392692c005a8160e9aeb1b7a2976b57f674393b40c0fff0079a6890dc63d1"} Jun 10 17:45:15.003090 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:15.003065 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mt5jr" Jun 10 17:45:15.995878 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:15.995835 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" event={"ID":"a56e596d-753a-4cce-808d-e8a9bf864b06","Type":"ContainerStarted","Data":"cb0ad4048b77950b4ee190531c57906abcf5e437b9cdecce1f60f0269a79f0ec"} Jun 10 17:45:16.014278 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:16.014234 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" podStartSLOduration=1.397508016 podStartE2EDuration="3.014221535s" totalImagesPullingTime="280.543464ms" totalInitContainerRuntime="1.336170055s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:45:16.012118638 +0000 UTC m=+825.980324678" watchObservedRunningTime="2026-06-10 17:45:16.014221535 +0000 UTC m=+825.982427574" Jun 10 17:45:23.389904 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:23.389869 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:23.390354 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:23.389924 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:23.402589 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:23.402562 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:24.045045 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:24.045012 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln" Jun 10 17:45:28.765449 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.765410 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x"] Jun 10 17:45:28.780063 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.780038 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:28.781065 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.781037 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x"] Jun 10 17:45:28.782342 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.782321 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 10 17:45:28.926315 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.926280 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:28.926315 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.926319 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:28.926527 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.926343 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:28.926527 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.926421 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:28.926527 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.926470 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:28.926527 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:28.926491 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvsgf\" (UniqueName: \"kubernetes.io/projected/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-kube-api-access-fvsgf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.027779 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.027688 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.027779 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.027732 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvsgf\" (UniqueName: \"kubernetes.io/projected/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-kube-api-access-fvsgf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.027779 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.027772 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.028018 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.027792 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.028018 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.027811 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.028018 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.027844 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.028250 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.028221 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.028372 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.028297 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.028432 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.028372 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.030142 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.030120 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.030365 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.030345 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.035640 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.035611 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvsgf\" (UniqueName: \"kubernetes.io/projected/3d980ac8-f807-4a88-96b8-e1edbc0c6eb5-kube-api-access-fvsgf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x\" (UID: \"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.091204 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.091166 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:45:29.223877 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:29.223848 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x"] Jun 10 17:45:29.226292 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:45:29.226258 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d980ac8_f807_4a88_96b8_e1edbc0c6eb5.slice/crio-839470b1fb38f735647bcc64dbe7d0f1d9907b61f0e7fe55b01156fddf4b5d64 WatchSource:0}: Error finding container 839470b1fb38f735647bcc64dbe7d0f1d9907b61f0e7fe55b01156fddf4b5d64: Status 404 returned error can't find the container with id 839470b1fb38f735647bcc64dbe7d0f1d9907b61f0e7fe55b01156fddf4b5d64 Jun 10 17:45:30.064778 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:30.064735 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" event={"ID":"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5","Type":"ContainerStarted","Data":"8c33c2516219a0a0797dce74b52b37b7001efd953658add19c9425f0b1ddadf7"} Jun 10 17:45:30.065162 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:30.064807 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" event={"ID":"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5","Type":"ContainerStarted","Data":"839470b1fb38f735647bcc64dbe7d0f1d9907b61f0e7fe55b01156fddf4b5d64"} Jun 10 17:45:32.676496 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.676465 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb"] Jun 10 17:45:32.679887 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.679867 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.682019 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.681969 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 10 17:45:32.690332 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.690309 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb"] Jun 10 17:45:32.758533 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.758503 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr4g9\" (UniqueName: \"kubernetes.io/projected/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-kube-api-access-xr4g9\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.758658 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.758548 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.758658 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.758581 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.758735 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.758704 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.758774 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.758746 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.758774 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.758764 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860169 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860138 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860323 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860175 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860323 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860247 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860323 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860263 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860323 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860277 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860323 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860311 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr4g9\" (UniqueName: \"kubernetes.io/projected/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-kube-api-access-xr4g9\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860684 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860660 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860760 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860727 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.860829 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.860790 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.862627 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.862608 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.863040 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.863019 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.867687 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.867668 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr4g9\" (UniqueName: \"kubernetes.io/projected/fa64eb26-a06d-4116-84cb-c3e7b1df5e23-kube-api-access-xr4g9\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb\" (UID: \"fa64eb26-a06d-4116-84cb-c3e7b1df5e23\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:32.991611 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:32.991534 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:45:33.131406 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:33.131383 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb"] Jun 10 17:45:33.133460 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:45:33.133430 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa64eb26_a06d_4116_84cb_c3e7b1df5e23.slice/crio-ef1daebed7c1b8aad21cd1b6122906fa40736a70c48334e4bec09def1854131a WatchSource:0}: Error finding container ef1daebed7c1b8aad21cd1b6122906fa40736a70c48334e4bec09def1854131a: Status 404 returned error can't find the container with id ef1daebed7c1b8aad21cd1b6122906fa40736a70c48334e4bec09def1854131a Jun 10 17:45:34.085465 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:34.085428 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" event={"ID":"fa64eb26-a06d-4116-84cb-c3e7b1df5e23","Type":"ContainerStarted","Data":"0e26b512040fae68b047f1d5bf4488e4871995ea433f533615182e9f54e225d8"} Jun 10 17:45:34.085918 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:34.085479 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" event={"ID":"fa64eb26-a06d-4116-84cb-c3e7b1df5e23","Type":"ContainerStarted","Data":"ef1daebed7c1b8aad21cd1b6122906fa40736a70c48334e4bec09def1854131a"} Jun 10 17:45:55.499440 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.499406 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs"] Jun 10 17:45:55.503449 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.503432 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.516986 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.516931 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 10 17:45:55.529778 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.529752 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs"] Jun 10 17:45:55.558319 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.558292 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.558418 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.558395 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.558466 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.558424 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.558561 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.558540 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.558615 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.558596 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.558651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.558637 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn6qz\" (UniqueName: \"kubernetes.io/projected/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-kube-api-access-hn6qz\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.659911 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.659878 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn6qz\" (UniqueName: \"kubernetes.io/projected/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-kube-api-access-hn6qz\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.660113 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.659931 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.660113 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.660011 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.660113 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.660042 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.660274 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.660174 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.660274 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.660215 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.660379 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.660364 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.660472 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.660445 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.660549 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.660530 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.662512 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.662488 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.662839 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.662823 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.667546 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.667522 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn6qz\" (UniqueName: \"kubernetes.io/projected/3028cb9d-27e6-4c42-93b7-6b1a4c01b75e-kube-api-access-hn6qz\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ldqxs\" (UID: \"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.812955 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.812850 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:45:55.947735 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:55.947713 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs"] Jun 10 17:45:55.949684 ip-10-0-138-237 kubenswrapper[2531]: W0610 17:45:55.949655 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3028cb9d_27e6_4c42_93b7_6b1a4c01b75e.slice/crio-421d8ab475374fc2f920a132b29baf4953d5015eba2fdd45486cb71b28013554 WatchSource:0}: Error finding container 421d8ab475374fc2f920a132b29baf4953d5015eba2fdd45486cb71b28013554: Status 404 returned error can't find the container with id 421d8ab475374fc2f920a132b29baf4953d5015eba2fdd45486cb71b28013554 Jun 10 17:45:56.185161 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:56.185074 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" event={"ID":"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e","Type":"ContainerStarted","Data":"6209219ce158bbc1c8b2b23d820d9135b49db4568833cf30e1d64a4e77170007"} Jun 10 17:45:56.185161 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:56.185122 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" event={"ID":"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e","Type":"ContainerStarted","Data":"421d8ab475374fc2f920a132b29baf4953d5015eba2fdd45486cb71b28013554"} Jun 10 17:45:58.196912 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:58.196877 2531 generic.go:356] "Generic (PLEG): container finished" podID="3028cb9d-27e6-4c42-93b7-6b1a4c01b75e" containerID="6209219ce158bbc1c8b2b23d820d9135b49db4568833cf30e1d64a4e77170007" exitCode=0 Jun 10 17:45:58.197285 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:58.196930 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" event={"ID":"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e","Type":"ContainerDied","Data":"6209219ce158bbc1c8b2b23d820d9135b49db4568833cf30e1d64a4e77170007"} Jun 10 17:45:59.204009 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:59.203975 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" event={"ID":"3028cb9d-27e6-4c42-93b7-6b1a4c01b75e","Type":"ContainerStarted","Data":"a3a8af2c777af6652e392309af27e7cfc94bb2f1327506e73a7742f49ee20cbd"} Jun 10 17:45:59.220238 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:45:59.220194 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" podStartSLOduration=2.690618289 podStartE2EDuration="4.220179706s" totalImagesPullingTime="206.902268ms" totalInitContainerRuntime="1.322659149s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:45:59.219129799 +0000 UTC m=+869.187335850" watchObservedRunningTime="2026-06-10 17:45:59.220179706 +0000 UTC m=+869.188385745" Jun 10 17:46:01.214779 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:01.214742 2531 generic.go:356] "Generic (PLEG): container finished" podID="8f55e024-38a9-4a81-998f-e0d937bc6ce4" containerID="148c09a3293084e826ee455beb1456fa5a1d7e4d2dd39938a402a0e21482b933" exitCode=0 Jun 10 17:46:01.215235 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:01.214812 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" event={"ID":"8f55e024-38a9-4a81-998f-e0d937bc6ce4","Type":"ContainerDied","Data":"148c09a3293084e826ee455beb1456fa5a1d7e4d2dd39938a402a0e21482b933"} Jun 10 17:46:02.222492 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:02.222458 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" event={"ID":"8f55e024-38a9-4a81-998f-e0d937bc6ce4","Type":"ContainerStarted","Data":"675c5b4c35e6fe4afbab8d191426112fb2cd2005bb65690ee17461096356e129"} Jun 10 17:46:02.240388 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:02.240345 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" podStartSLOduration=2.690280963 podStartE2EDuration="1m1.240330834s" totalImagesPullingTime="4.443548761s" totalInitContainerRuntime="54.10650111s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:45:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:46:02.238645306 +0000 UTC m=+872.206851344" watchObservedRunningTime="2026-06-10 17:46:02.240330834 +0000 UTC m=+872.208536873" Jun 10 17:46:05.813384 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:05.813342 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:46:05.813790 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:05.813405 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:46:05.826114 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:05.826083 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:46:06.252221 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:06.252200 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ldqxs" Jun 10 17:46:11.911357 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:11.911318 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:46:11.911738 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:11.911385 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:46:11.923854 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:11.923830 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:46:12.281572 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:12.281548 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks" Jun 10 17:46:22.317811 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:22.317776 2531 generic.go:356] "Generic (PLEG): container finished" podID="3d980ac8-f807-4a88-96b8-e1edbc0c6eb5" containerID="8c33c2516219a0a0797dce74b52b37b7001efd953658add19c9425f0b1ddadf7" exitCode=0 Jun 10 17:46:22.318204 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:22.317850 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" event={"ID":"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5","Type":"ContainerDied","Data":"8c33c2516219a0a0797dce74b52b37b7001efd953658add19c9425f0b1ddadf7"} Jun 10 17:46:23.324682 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:23.324647 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" event={"ID":"3d980ac8-f807-4a88-96b8-e1edbc0c6eb5","Type":"ContainerStarted","Data":"9ecd03ebbbf62948bae51c32d962dd64d613b7e28ea5248dd9f84bb9655355ea"} Jun 10 17:46:23.342051 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:23.342004 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" podStartSLOduration=2.371659299 podStartE2EDuration="55.341988819s" totalImagesPullingTime="188.049218ms" totalInitContainerRuntime="52.782280302s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:46:23.339977857 +0000 UTC m=+893.308183898" watchObservedRunningTime="2026-06-10 17:46:23.341988819 +0000 UTC m=+893.310194858" Jun 10 17:46:29.092164 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:29.092109 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:46:29.092651 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:29.092180 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:46:29.104617 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:29.104592 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:46:29.368931 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:29.368849 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x" Jun 10 17:46:30.467738 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:30.467707 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:46:30.468323 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:46:30.467769 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:48:20.889401 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:48:20.889363 2531 generic.go:356] "Generic (PLEG): container finished" podID="fa64eb26-a06d-4116-84cb-c3e7b1df5e23" containerID="0e26b512040fae68b047f1d5bf4488e4871995ea433f533615182e9f54e225d8" exitCode=0 Jun 10 17:48:20.889818 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:48:20.889437 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" event={"ID":"fa64eb26-a06d-4116-84cb-c3e7b1df5e23","Type":"ContainerDied","Data":"0e26b512040fae68b047f1d5bf4488e4871995ea433f533615182e9f54e225d8"} Jun 10 17:48:21.896023 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:48:21.895987 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" event={"ID":"fa64eb26-a06d-4116-84cb-c3e7b1df5e23","Type":"ContainerStarted","Data":"61e3e168b82579eb3e32fd0a63a5ea9405adfbca27f1f644a372b80e0979bdd4"} Jun 10 17:48:21.912418 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:48:21.912373 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" podStartSLOduration=2.842773656 podStartE2EDuration="2m49.912357939s" totalImagesPullingTime="196.107384ms" totalInitContainerRuntime="2m46.873476899s" isStatefulPod=false podCreationTimestamp="2026-06-10 17:45:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 17:48:21.91147388 +0000 UTC m=+1011.879679920" watchObservedRunningTime="2026-06-10 17:48:21.912357939 +0000 UTC m=+1011.880563978" Jun 10 17:48:22.992203 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:48:22.992167 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:48:22.992643 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:48:22.992231 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:48:23.004878 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:48:23.004855 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:48:23.917351 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:48:23.917318 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb" Jun 10 17:49:53.243581 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.243546 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bdd4f677f-r26bp"] Jun 10 17:49:53.244085 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.243765 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-bdd4f677f-r26bp" podUID="82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" containerName="manager" containerID="cri-o://d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119" gracePeriod=30 Jun 10 17:49:53.499711 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.499693 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:49:53.607685 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.607659 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkf88\" (UniqueName: \"kubernetes.io/projected/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-kube-api-access-nkf88\") pod \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " Jun 10 17:49:53.607853 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.607700 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-webhook-cert\") pod \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " Jun 10 17:49:53.607853 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.607734 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-odh-trusted-ca-bundle\") pod \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\" (UID: \"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9\") " Jun 10 17:49:53.608111 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.608083 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-odh-trusted-ca-bundle" (OuterVolumeSpecName: "odh-trusted-ca-bundle") pod "82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" (UID: "82ebbadc-bab4-4d71-b51f-c39eb70fa4d9"). InnerVolumeSpecName "odh-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 17:49:53.609834 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.609812 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" (UID: "82ebbadc-bab4-4d71-b51f-c39eb70fa4d9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 17:49:53.609948 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.609917 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-kube-api-access-nkf88" (OuterVolumeSpecName: "kube-api-access-nkf88") pod "82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" (UID: "82ebbadc-bab4-4d71-b51f-c39eb70fa4d9"). InnerVolumeSpecName "kube-api-access-nkf88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 17:49:53.708846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.708815 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkf88\" (UniqueName: \"kubernetes.io/projected/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-kube-api-access-nkf88\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:49:53.708846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.708840 2531 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-webhook-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:49:53.708846 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:53.708851 2531 reconciler_common.go:299] "Volume detached for volume \"odh-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9-odh-trusted-ca-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 17:49:54.346555 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.346522 2531 generic.go:356] "Generic (PLEG): container finished" podID="82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" containerID="d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119" exitCode=0 Jun 10 17:49:54.346555 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.346568 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bdd4f677f-r26bp" Jun 10 17:49:54.347067 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.346604 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bdd4f677f-r26bp" event={"ID":"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9","Type":"ContainerDied","Data":"d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119"} Jun 10 17:49:54.347067 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.346659 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bdd4f677f-r26bp" event={"ID":"82ebbadc-bab4-4d71-b51f-c39eb70fa4d9","Type":"ContainerDied","Data":"5713ede8055a040ed8a1544a4fd6bbfbe82007a218c1bafa91f65a863619822d"} Jun 10 17:49:54.347067 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.346682 2531 scope.go:117] "RemoveContainer" containerID="d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119" Jun 10 17:49:54.356368 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.356352 2531 scope.go:117] "RemoveContainer" containerID="d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119" Jun 10 17:49:54.356592 ip-10-0-138-237 kubenswrapper[2531]: E0610 17:49:54.356574 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119\": container with ID starting with d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119 not found: ID does not exist" containerID="d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119" Jun 10 17:49:54.356654 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.356603 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119"} err="failed to get container status \"d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119\": rpc error: code = NotFound desc = could not find container \"d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119\": container with ID starting with d9e0c9d21bd74ad737fcce91e3517293851558f4eb4046e00e1f8641e910e119 not found: ID does not exist" Jun 10 17:49:54.366322 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.366302 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bdd4f677f-r26bp"] Jun 10 17:49:54.370064 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.370044 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-bdd4f677f-r26bp"] Jun 10 17:49:54.472338 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:49:54.472309 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" path="/var/lib/kubelet/pods/82ebbadc-bab4-4d71-b51f-c39eb70fa4d9/volumes" Jun 10 17:51:30.503020 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:51:30.502995 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:51:30.504045 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:51:30.504025 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:56:30.534592 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:56:30.534565 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 17:56:30.535350 ip-10-0-138-237 kubenswrapper[2531]: I0610 17:56:30.535318 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 18:00:24.362764 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.362726 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs"] Jun 10 18:00:24.363241 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.362929 2531 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" podUID="b7585638-a85e-49d9-b0ce-54c80e6c562c" containerName="manager" containerID="cri-o://9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992" gracePeriod=10 Jun 10 18:00:24.621020 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.620959 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 18:00:24.659278 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.659247 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkqwn\" (UniqueName: \"kubernetes.io/projected/b7585638-a85e-49d9-b0ce-54c80e6c562c-kube-api-access-kkqwn\") pod \"b7585638-a85e-49d9-b0ce-54c80e6c562c\" (UID: \"b7585638-a85e-49d9-b0ce-54c80e6c562c\") " Jun 10 18:00:24.659418 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.659288 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7585638-a85e-49d9-b0ce-54c80e6c562c-extensions-socket-volume\") pod \"b7585638-a85e-49d9-b0ce-54c80e6c562c\" (UID: \"b7585638-a85e-49d9-b0ce-54c80e6c562c\") " Jun 10 18:00:24.659701 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.659670 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7585638-a85e-49d9-b0ce-54c80e6c562c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b7585638-a85e-49d9-b0ce-54c80e6c562c" (UID: "b7585638-a85e-49d9-b0ce-54c80e6c562c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 18:00:24.661599 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.661573 2531 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7585638-a85e-49d9-b0ce-54c80e6c562c-kube-api-access-kkqwn" (OuterVolumeSpecName: "kube-api-access-kkqwn") pod "b7585638-a85e-49d9-b0ce-54c80e6c562c" (UID: "b7585638-a85e-49d9-b0ce-54c80e6c562c"). InnerVolumeSpecName "kube-api-access-kkqwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 18:00:24.760462 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.760427 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkqwn\" (UniqueName: \"kubernetes.io/projected/b7585638-a85e-49d9-b0ce-54c80e6c562c-kube-api-access-kkqwn\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 18:00:24.760462 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:24.760455 2531 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7585638-a85e-49d9-b0ce-54c80e6c562c-extensions-socket-volume\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\"" Jun 10 18:00:25.363809 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.363771 2531 generic.go:356] "Generic (PLEG): container finished" podID="b7585638-a85e-49d9-b0ce-54c80e6c562c" containerID="9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992" exitCode=0 Jun 10 18:00:25.363809 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.363811 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" Jun 10 18:00:25.364350 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.363842 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" event={"ID":"b7585638-a85e-49d9-b0ce-54c80e6c562c","Type":"ContainerDied","Data":"9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992"} Jun 10 18:00:25.364350 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.363894 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs" event={"ID":"b7585638-a85e-49d9-b0ce-54c80e6c562c","Type":"ContainerDied","Data":"4a33ffedb0e6df332362c04b01344c616ecbfc0793c72fefacddf1e97af26c93"} Jun 10 18:00:25.364350 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.363927 2531 scope.go:117] "RemoveContainer" containerID="9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992" Jun 10 18:00:25.377985 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.377960 2531 scope.go:117] "RemoveContainer" containerID="9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992" Jun 10 18:00:25.378266 ip-10-0-138-237 kubenswrapper[2531]: E0610 18:00:25.378232 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992\": container with ID starting with 9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992 not found: ID does not exist" containerID="9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992" Jun 10 18:00:25.378359 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.378285 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992"} err="failed to get container status \"9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992\": rpc error: code = NotFound desc = could not find container \"9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992\": container with ID starting with 9f461f8ec4b086af9912917475bb6fb181b7ee3fc960d2ba5394b206eba85992 not found: ID does not exist" Jun 10 18:00:25.396229 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.396209 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs"] Jun 10 18:00:25.404519 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:25.404493 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vdwfs"] Jun 10 18:00:26.470531 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:00:26.470495 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7585638-a85e-49d9-b0ce-54c80e6c562c" path="/var/lib/kubelet/pods/b7585638-a85e-49d9-b0ce-54c80e6c562c/volumes" Jun 10 18:01:30.566875 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:01:30.566845 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 18:01:30.567901 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:01:30.567877 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 18:06:30.598269 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:06:30.598236 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 18:06:30.600728 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:06:30.600552 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 18:11:30.627775 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:30.627660 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 18:11:30.632117 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:30.632099 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k9jjg_10ca2f89-938d-4ed7-a81d-cf2f0f85d945/ovn-acl-logging/0.log" Jun 10 18:11:43.128146 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:43.128117 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7474f67696-477n8_577450d8-2a8e-4754-ba87-02c2053599e7/nginx/0.log" Jun 10 18:11:43.581898 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:43.581869 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-784fdbf859-82rcr_fac72386-d677-4971-a124-be3582bde7c0/maas-api/0.log" Jun 10 18:11:44.509121 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:44.509092 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55bf9956cc-gv2bh_cdb0f390-27ea-4e36-a5d4-2c58145a44e0/copy-manifests/0.log" Jun 10 18:11:44.516216 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:44.516194 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55bf9956cc-gv2bh_cdb0f390-27ea-4e36-a5d4-2c58145a44e0/manager/0.log" Jun 10 18:11:44.629278 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:44.629245 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-zpn48_5f4d99a9-9336-45a9-b4cd-899ed1e3477d/postgres/0.log" Jun 10 18:11:46.233023 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:46.232992 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-pjl5g_15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd/kuadrant-console-plugin/0.log" Jun 10 18:11:46.352712 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:46.352685 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b2pvh_7042b236-9910-4ce7-9dab-2278053d89a5/registry-server/0.log" Jun 10 18:11:47.069736 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:47.069703 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795cfcqq_c9ffd22f-8d03-45ec-a2e0-867870c17f66/istio-proxy/0.log" Jun 10 18:11:47.513299 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:47.513268 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/1.log" Jun 10 18:11:47.749026 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:47.748996 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-8wd86_f75622db-3970-4410-8962-fe8f3bde954a/payload-processing/0.log" Jun 10 18:11:48.206685 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.206657 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks_8f55e024-38a9-4a81-998f-e0d937bc6ce4/storage-initializer/0.log" Jun 10 18:11:48.213789 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.213769 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-bmlks_8f55e024-38a9-4a81-998f-e0d937bc6ce4/main/0.log" Jun 10 18:11:48.322088 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.322056 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-mt5jr_6a6e14bb-4624-41a0-bdbf-45a6805a809e/storage-initializer/0.log" Jun 10 18:11:48.329880 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.329853 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-mt5jr_6a6e14bb-4624-41a0-bdbf-45a6805a809e/main/0.log" Jun 10 18:11:48.441858 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.441834 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln_a56e596d-753a-4cce-808d-e8a9bf864b06/storage-initializer/0.log" Jun 10 18:11:48.449054 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.449035 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-j55ln_a56e596d-753a-4cce-808d-e8a9bf864b06/main/0.log" Jun 10 18:11:48.556370 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.556344 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x_3d980ac8-f807-4a88-96b8-e1edbc0c6eb5/storage-initializer/0.log" Jun 10 18:11:48.563592 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.563568 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c6rw8x_3d980ac8-f807-4a88-96b8-e1edbc0c6eb5/main/0.log" Jun 10 18:11:48.680594 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.680553 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb_fa64eb26-a06d-4116-84cb-c3e7b1df5e23/storage-initializer/0.log" Jun 10 18:11:48.688685 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.688666 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-lrpdb_fa64eb26-a06d-4116-84cb-c3e7b1df5e23/main/0.log" Jun 10 18:11:48.799573 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.799537 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-ldqxs_3028cb9d-27e6-4c42-93b7-6b1a4c01b75e/storage-initializer/0.log" Jun 10 18:11:48.806989 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:48.806915 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-ldqxs_3028cb9d-27e6-4c42-93b7-6b1a4c01b75e/main/0.log" Jun 10 18:11:56.092624 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:56.092594 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6wslj_70f889fb-10f8-4bc7-a0a2-42e0ab79dc87/global-pull-secret-syncer/0.log" Jun 10 18:11:56.246464 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:56.246435 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rk5fq_6bc8234a-907c-41cd-9900-f4183678e844/konnectivity-agent/0.log" Jun 10 18:11:56.355617 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:11:56.355549 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-237.ec2.internal_193fc2f766e2d337515229dea4784854/haproxy/0.log" Jun 10 18:12:01.507300 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:01.507268 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-pjl5g_15f81e6c-bbd7-4d60-a5e8-b372ca4b97dd/kuadrant-console-plugin/0.log" Jun 10 18:12:01.544498 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:01.544468 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b2pvh_7042b236-9910-4ce7-9dab-2278053d89a5/registry-server/0.log" Jun 10 18:12:03.690213 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:03.690183 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-nh54r_3ddba959-6c32-43d7-9db4-40730e43f44d/monitoring-plugin/0.log" Jun 10 18:12:03.883092 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:03.883068 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ttbfl_a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2/node-exporter/0.log" Jun 10 18:12:03.904020 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:03.903997 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ttbfl_a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2/kube-rbac-proxy/0.log" Jun 10 18:12:03.925689 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:03.925669 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ttbfl_a8a3477b-eaa4-4c6d-9710-f1c7be1e91d2/init-textfile/0.log" Jun 10 18:12:04.358485 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:04.358411 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-549b58c766-hrxj2_eb94fafe-a14f-4db9-9e02-91572e0a8f9a/telemeter-client/0.log" Jun 10 18:12:04.379680 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:04.379655 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-549b58c766-hrxj2_eb94fafe-a14f-4db9-9e02-91572e0a8f9a/reload/0.log" Jun 10 18:12:04.399861 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:04.399842 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-549b58c766-hrxj2_eb94fafe-a14f-4db9-9e02-91572e0a8f9a/kube-rbac-proxy/0.log" Jun 10 18:12:07.251802 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:07.251759 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65447df49f-nnlzl_d074da7d-d94c-4bdd-86f6-645c2aea4770/console/0.log" Jun 10 18:12:07.281409 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:07.281378 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-45hxm_ab36f2dd-c32b-40db-ba57-9bda6babeff8/download-server/0.log" Jun 10 18:12:08.155093 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.155055 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2"] Jun 10 18:12:08.155426 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.155413 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" containerName="manager" Jun 10 18:12:08.155475 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.155428 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" containerName="manager" Jun 10 18:12:08.155475 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.155438 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7585638-a85e-49d9-b0ce-54c80e6c562c" containerName="manager" Jun 10 18:12:08.155475 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.155443 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7585638-a85e-49d9-b0ce-54c80e6c562c" containerName="manager" Jun 10 18:12:08.155569 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.155502 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7585638-a85e-49d9-b0ce-54c80e6c562c" containerName="manager" Jun 10 18:12:08.155569 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.155511 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ebbadc-bab4-4d71-b51f-c39eb70fa4d9" containerName="manager" Jun 10 18:12:08.158388 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.158367 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.160418 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.160402 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-58lq9\"/\"kube-root-ca.crt\"" Jun 10 18:12:08.161189 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.161173 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-58lq9\"/\"default-dockercfg-fz7rm\"" Jun 10 18:12:08.161291 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.161223 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-58lq9\"/\"openshift-service-ca.crt\"" Jun 10 18:12:08.166658 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.166631 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2"] Jun 10 18:12:08.336118 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.336076 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-podres\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.336520 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.336133 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-lib-modules\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.336520 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.336257 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-proc\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.336520 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.336288 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xls9r\" (UniqueName: \"kubernetes.io/projected/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-kube-api-access-xls9r\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.336520 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.336314 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-sys\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437220 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437126 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-proc\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437220 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437167 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xls9r\" (UniqueName: \"kubernetes.io/projected/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-kube-api-access-xls9r\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437220 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437196 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-sys\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437478 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437245 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-proc\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437478 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437248 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-podres\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437478 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437318 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-lib-modules\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437478 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437328 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-sys\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437478 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437350 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-podres\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.437478 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.437411 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-lib-modules\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.445383 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.445354 2531 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xls9r\" (UniqueName: \"kubernetes.io/projected/f85c0c2d-d3c3-4ee8-ad95-a87b81e99221-kube-api-access-xls9r\") pod \"perf-node-gather-daemonset-8mrf2\" (UID: \"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221\") " pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.469660 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.469637 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:08.601716 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.601682 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2"] Jun 10 18:12:08.603039 ip-10-0-138-237 kubenswrapper[2531]: W0610 18:12:08.603010 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf85c0c2d_d3c3_4ee8_ad95_a87b81e99221.slice/crio-c5cd0b9f6299bb262cb463f2299fb6bcb502fd0d530018adcd0265be521dfa91 WatchSource:0}: Error finding container c5cd0b9f6299bb262cb463f2299fb6bcb502fd0d530018adcd0265be521dfa91: Status 404 returned error can't find the container with id c5cd0b9f6299bb262cb463f2299fb6bcb502fd0d530018adcd0265be521dfa91 Jun 10 18:12:08.661386 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:08.661358 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" event={"ID":"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221","Type":"ContainerStarted","Data":"c5cd0b9f6299bb262cb463f2299fb6bcb502fd0d530018adcd0265be521dfa91"} Jun 10 18:12:09.170636 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:09.170607 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2227b_6f80cc11-56c3-4148-bd3f-17559675eaed/dns/0.log" Jun 10 18:12:09.193826 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:09.193798 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2227b_6f80cc11-56c3-4148-bd3f-17559675eaed/kube-rbac-proxy/0.log" Jun 10 18:12:09.364802 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:09.364769 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n8cmd_04b03051-fc01-4dac-a94b-b7ddc921fdf2/dns-node-resolver/0.log" Jun 10 18:12:09.668213 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:09.668171 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" event={"ID":"f85c0c2d-d3c3-4ee8-ad95-a87b81e99221","Type":"ContainerStarted","Data":"cd2fcc26d9dd2974f179e6a38711eff9e72fd81a4502fb099420749d1fe3f3ef"} Jun 10 18:12:09.668411 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:09.668268 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:09.685467 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:09.685423 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" podStartSLOduration=1.685409957 podStartE2EDuration="1.685409957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 18:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 18:12:09.683820117 +0000 UTC m=+2439.652026172" watchObservedRunningTime="2026-06-10 18:12:09.685409957 +0000 UTC m=+2439.653615996" Jun 10 18:12:09.975159 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:09.975083 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vsqlg_111e361e-8521-49d3-9685-9b21257ea3d3/node-ca/0.log" Jun 10 18:12:10.931688 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:10.931658 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795cfcqq_c9ffd22f-8d03-45ec-a2e0-867870c17f66/istio-proxy/0.log" Jun 10 18:12:11.090482 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:11.090446 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/0.log" Jun 10 18:12:11.119077 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:11.119046 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/1.log" Jun 10 18:12:11.220443 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:11.220373 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-8wd86_f75622db-3970-4410-8962-fe8f3bde954a/payload-processing/0.log" Jun 10 18:12:11.894317 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:11.894291 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tw5jz_6fc5ad1d-f684-4158-b1e1-bac37539d5f9/serve-healthcheck-canary/0.log" Jun 10 18:12:12.424236 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:12.424206 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9kznn_0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7/kube-rbac-proxy/0.log" Jun 10 18:12:12.444234 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:12.444210 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9kznn_0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7/exporter/0.log" Jun 10 18:12:12.468237 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:12.468214 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9kznn_0dd8a6cd-4fbb-4c25-a36e-cfcda319b7e7/extractor/0.log" Jun 10 18:12:14.834268 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:14.834228 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7474f67696-477n8_577450d8-2a8e-4754-ba87-02c2053599e7/nginx/0.log" Jun 10 18:12:15.000067 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:15.000036 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-784fdbf859-82rcr_fac72386-d677-4971-a124-be3582bde7c0/maas-api/0.log" Jun 10 18:12:15.458761 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:15.458728 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55bf9956cc-gv2bh_cdb0f390-27ea-4e36-a5d4-2c58145a44e0/manager/0.log" Jun 10 18:12:15.482178 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:15.482151 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55bf9956cc-gv2bh_cdb0f390-27ea-4e36-a5d4-2c58145a44e0/copy-manifests/0.log" Jun 10 18:12:15.506832 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:15.506805 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-zpn48_5f4d99a9-9336-45a9-b4cd-899ed1e3477d/postgres/0.log" Jun 10 18:12:15.683450 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:15.683418 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-58lq9/perf-node-gather-daemonset-8mrf2" Jun 10 18:12:17.136898 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:17.136867 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6648fffc94-nttzx_caf6a2d1-1415-4a88-8333-3555a330c937/manager/0.log" Jun 10 18:12:17.164048 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:17.164028 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rl8v7_4b7b3d77-f1bf-4f1f-b626-2d54d51cdf4c/openshift-lws-operator/0.log" Jun 10 18:12:21.503865 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:21.503834 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795cfcqq_c9ffd22f-8d03-45ec-a2e0-867870c17f66/istio-proxy/0.log" Jun 10 18:12:21.616832 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:21.616806 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/0.log" Jun 10 18:12:21.651265 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:21.651231 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/1.log" Jun 10 18:12:21.725410 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:21.725381 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-8wd86_f75622db-3970-4410-8962-fe8f3bde954a/payload-processing/0.log" Jun 10 18:12:22.956646 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:22.956617 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795cfcqq_c9ffd22f-8d03-45ec-a2e0-867870c17f66/istio-proxy/0.log" Jun 10 18:12:23.068324 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:23.068288 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/0.log" Jun 10 18:12:23.101404 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:23.101370 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/1.log" Jun 10 18:12:23.181419 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:23.181390 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-8wd86_f75622db-3970-4410-8962-fe8f3bde954a/payload-processing/0.log" Jun 10 18:12:24.276043 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.276012 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zd59_f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8/kube-multus-additional-cni-plugins/0.log" Jun 10 18:12:24.297123 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.297095 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zd59_f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8/egress-router-binary-copy/0.log" Jun 10 18:12:24.318303 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.318281 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zd59_f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8/cni-plugins/0.log" Jun 10 18:12:24.337548 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.337525 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zd59_f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8/bond-cni-plugin/0.log" Jun 10 18:12:24.358572 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.358551 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zd59_f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8/routeoverride-cni/0.log" Jun 10 18:12:24.378337 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.378316 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zd59_f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8/whereabouts-cni-bincopy/0.log" Jun 10 18:12:24.398292 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.398271 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2zd59_f8d5a1bc-53b2-4935-bcf0-23ffdb236dc8/whereabouts-cni/0.log" Jun 10 18:12:24.755622 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.755593 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hskvc_02074e9f-b06c-412e-beee-54138e398848/kube-multus/0.log" Jun 10 18:12:24.805442 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.805416 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795cfcqq_c9ffd22f-8d03-45ec-a2e0-867870c17f66/istio-proxy/0.log" Jun 10 18:12:24.905015 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.904991 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/0.log" Jun 10 18:12:24.911989 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.911969 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f5f66_d9f00b72-1079-4ead-a63f-5761b40d3166/network-metrics-daemon/0.log" Jun 10 18:12:24.933523 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.933506 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f5f66_d9f00b72-1079-4ead-a63f-5761b40d3166/kube-rbac-proxy/0.log" Jun 10 18:12:24.936223 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:24.936210 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-98d67_a533ebf2-5111-41c2-8322-c7a463e38275/istio-proxy/1.log" Jun 10 18:12:25.014304 ip-10-0-138-237 kubenswrapper[2531]: I0610 18:12:25.014251 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-8wd86_f75622db-3970-4410-8962-fe8f3bde954a/payload-processing/0.log"