KUBEFLOW_SKIP_INSTALL is set. Skipping SDK installation to use the version baked into the notebook image. Installed kernelspec python3 in /opt/app-root/src/.local/share/jupyter/kernels/python3 Input Notebook: /opt/app-root/notebooks/rhai_features.ipynb Output Notebook: /opt/app-root/src/out.ipynb [IPKernelApp] WARNING | Kernel is running over TCP without encryption. All communication (including code and outputs) is sent in plain text and is susceptible to eavesdropping. Use IPC transport or launch with kernel manager-provisioned CurveZMQ keys to enable transport encryption. Executing notebook with kernel: python3 Executing Cell 1--------------------------------------- Ending Cell 1------------------------------------------ Executing Cell 2--------------------------------------- Ending Cell 2------------------------------------------ Executing Cell 3--------------------------------------- Trainer control-plane version info is not available: unable to read 'kubeflow_trainer_version' from ConfigMap 'kubeflow-trainer-public' in namespace 'kubeflow-system': (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Audit-Id': '917907a4-521b-4e9b-b16f-62584f285db2', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '77f5472a-d825-4035-bcb9-80563d58e8d5', 'X-Kubernetes-Pf-Prioritylevel-Uid': '750844ff-09bd-4f0c-a196-970589515e39', 'Date': 'Fri, 23 Oct 2026 04:12:16 GMT', 'Content-Length': '388'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps \"kubeflow-trainer-public\" is forbidden: User \"system:serviceaccount:redhat-ods-applications:trainer-e2e\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"kubeflow-system\"","reason":"Forbidden","details":{"name":"kubeflow-trainer-public","kind":"configmaps"},"code":403} . If your Trainer control-plane is installed in a different namespace, set the KUBEFLOW_SYSTEM_NAMESPACE environment variable accordingly. API: https://a4025c7ccf93e49d0946af1abba28101-bdc626a586a91815.elb.us-east-2.amazonaws.com:6443 Namespace: test-ns-mw5hw PVC: pvc-bcr6r TrainerClient initialized Ending Cell 3------------------------------------------ Executing Cell 4--------------------------------------- Got runtime: torch-distributed-cpu Ending Cell 4------------------------------------------ Executing Cell 5--------------------------------------- HuggingFace mode: training pods will download directly from HF Hub Ending Cell 5------------------------------------------ Executing Cell 6--------------------------------------- Progression tracking: True JIT Checkpoint: False Training config: num_nodes=1, num_gpus_per_node=1, gpu_type=cpu CPU mode: no GPU requested TRAINJOB_NAME: n500f990c367 Ending Cell 6------------------------------------------ Executing Cell 7--------------------------------------- Training job final status: Complete Ending Cell 7------------------------------------------ Executing Cell 8--------------------------------------- Step: node-0, Status: Succeeded, Devices: cpu x 1 Ending Cell 8------------------------------------------ Executing Cell 9--------------------------------------- [Kubeflow] Initializing progression tracking [Kubeflow] Progression tracking enabled CPU mode forced via GPU_TYPE=cpu Detected accelerator: CPU (forced) Using backend: gloo, fp16=False, bf16=False Distributed: WORLD_SIZE=1, RANK=0, LOCAL_RANK=0 HuggingFace mode: model=distilgpt2, dataset=yahma/alpaca-cleaned Downloading model from HuggingFace: distilgpt2 Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. Loading weights: 0%| | 0/76 [00:00